o ɶdf@sddlmZddlmZddlmZddlmZmZmZddl Z dZ d Z d Z d Z e e e e gZGd d d eZGdddeZGdddeZGdddeZGdddeZGdddeZGdddeZdS)) ServiceError)retry_checkers)retry_sleep_utils)%should_record_body_position_for_retryrecord_body_position_for_rewind rewind_bodyNZ full_jitterZ equal_jitterZ"full_jitter_with_equal_on_throttledecorrelated_jitterc@sTeZdZdZddZdddZddZdd d Zd d ZddZ ddZ ddZ dS)RetryStrategyBuildera A class which can build a retry strategy based on provided criteria. Criteria can be provided at construction time or afterwards via using the ``add_*`` (to add/enable criteria) and ``no_*`` (to disable/remove criteria) methods. Trying to build a strategy when there are no enabled checks will result in a :py:class:`oci.retry.NoneRetryStrategy` being produced. This builder is intended as a convenience, but callers are also able to bypass this and construct retry strategies directly. cKs|dd|_|dd|_|dd|_|dd|_|dd|_|d tj|_|d d|_ |d d |_ |d d|_ |dd|_ |dd |_ t|_d|vri|dtvrbtdt|d|_dSdS)a Creates a new builder and initializes it based on any provided parameters. :param Boolean max_attempts_check (optional): Whether to enable a check that we don't exceed a certain number of attempts. If not provided this defaults to True (with a default max-attempts = 8) :param Boolean service_error_check (optional): Whether to enable a check that will retry on connection errors, timeouts and service errors which match given combinations of HTTP statuses and textual error codes. If not provided this defaults to True (i.e. this check will be done) :param Boolean total_elapsed_time_check (optional): Whether to enable a check that we don't exceed a certain amount of time when retrying. This is intended for scenarios such as "keep retrying for 5 minutes". If not provided this defaults to True (i.e. this check will be done with the total elapsed time of 600 seconds) :param int max_attempts (optional): If we are checking that we don't exceed a certain number of attempts, what that number of attempts should be. This only applies if we are performing a check on the maximum number of attempts and will be ignored otherwise. If we are performing a check on the maximum number of attempts and this value is not provided, we will default to a maximum of 8 attempts :param int total_elapsed_time_seconds (optional): If we are checking that we don't exceed a certain amount of time when retrying, what that amount of time should be (in seconds). This only applies if we are performing a check on the total elapsed time and will be ignored otherwise. If we are performing a check on the total elapsed time and this value is not provided, we will default to 600 seconds (10 minutes) :param dict service_error_retry_config (optional): If we are checking on service errors, we can configure what HTTP statuses (e.g. 429) to retry on and, optionally, whether the textual code (e.g. TooManyRequests) matches a given value. This is a dictionary where the key is an integer representing the HTTP status, and the value is a list(str) where we will test if the textual code in the service error is a member of the list. If an empty list is provided, then only the numeric status is checked for retry purposes. If we are performing a check on service errors and this value is not provided, then by default we will retry on HTTP 409/IncorrectState, 429's (throttles) without any textual code check. :param Boolean service_error_retry_on_any_5xx (optional): If we are checking on service errors, whether to retry on any HTTP 5xx received from the service. If we are performing a check on service errors and this value is not provided, it defaults to True (retry on any 5xx except 501) :param int retry_base_sleep_time_seconds (optional): For exponential backoff with jitter, the base time to use in our retry calculation in seconds. If not provided, this value defaults to 1 second :param int retry_exponential_growth_factor (optional): For exponential backoff with jitter, the exponent which we will raise to the power of the number of attempts. If not provided, this value defaults to 2 :param int retry_max_wait_between_calls_seconds (optional): For exponential backoff with jitter, the maximum amount of time to wait between retries. If not provided, this value defaults to 30 seconds :param int decorrelated_jitter (optional): The random De-correlated jitter value in seconds (default 1) to be used when using the backoff_type BACKOFF_DECORRELATED_JITTER_VALUE :param str backoff_type (optional): The type of backoff we want to do (e.g. full jitter). The convenience constants in this retry module: ``BACKOFF_DECORRELATED_JITTER_VALUE``, ``BACKOFF_FULL_JITTER_VALUE``,`BACKOFF_EQUAL_JITTER_VALUE``, and ``BACKOFF_FULL_JITTER_EQUAL_ON_THROTTLE_VALUE`` can be used as values here. If no value is specified then the value ``BACKOFF_DECORRELATED_JITTER_VALUE`` will be used. This will use exponential backoff and a random de-correlated jitter. max_attempts_checkTservice_error_checktotal_elapsed_time_check max_attemptstotal_elapsed_time_secondsXservice_error_retry_configservice_error_retry_on_any_5xxretry_base_sleep_time_secondsrretry_exponential_growth_factorr$retry_max_wait_between_calls_secondsr backoff_typezExponentialBackoffWithFullJitterEqualForThrottlesRetryStrategyrrr)r r3r7r"r"r#get_retry_strategysf     z'RetryStrategyBuilder.get_retry_strategyN)r)r) __name__ __module__ __qualname____doc__r$r'r,r.r/r0r1rBr"r"r"r#r s X  r c@seZdZdZddZdS)r8z( A strategy that does not retry cOs||i|S)5 Calls the function given by func_ref. Any positional (``*func_args``) and keyword (``**func_kwargs``) arguments are passed as-is to func_ref. :param function func_ref: The function that we should call with retries :return: the result of calling func_ref r")r func_ref func_args func_kwargsr"r"r#make_retrying_calls z$NoneRetryStrategy.make_retrying_callN)rCrDrErFrKr"r"r"r#r8s r8c@s0eZdZdZddZddZddZdd Zd S) #ExponentialBackoffRetryStrategyBasea A base retry strategy from which other retry strategies inherit. Implementors can create a subclass of this to define their own retry logic. This is primarily useful when an implementor wishes to customize the sleep strategy used - to customize the checking logic on whether a retry should be done, implementors can define their own subclasses of :py:class:`oci.retry.retry_checkers.BaseRetryChecker` and provide this in the checker container. cKs"||_||_||_||_d|_dS)a Creates a new instance of an exponential backoff with full jitter retry strategy. :param int base_sleep_time_seconds: The base amount to sleep by, in seconds :param int exponent_growth_factor: The exponent part of our backoff. We will raise take this value and raising it to the power of attemps and then multiply this with base_sleep_time_seconds :param int max_wait_between_calls_seconds: The maximum time we will wait between calls, in seconds :param retry_checkers.RetryCheckerContainer checker_container: The checks to run to determine whether a failed call should be retried N)r4r5r6r3circuit_breaker_callbackr r4r5r6r7r!r"r"r#r$s  z,ExponentialBackoffRetryStrategyBase.__init__c Osd}d}ddd}d}t}t|fi|} d} |jj} | D] } t| tjur+| j}qt|ddrE|j} t| dd|d<t| dd|d<|dr[|dr[|d d ||j |j |rz-| rit |d \} }||i|}t|}|dr|dr|d d ||WSty}zPt|d d}t|d d}|d7}| r|jj||t||jdr|||| r| rt|d |sƂ|dr|dr|d d |||nWYd}~nd}~ww|s]dSdS)rGTNF)loggerdebugr__self__rOrPz\Retry policy to use: MaximumNumberAttempts={}, MaxSleepBetween={}, ExponentialBackoffBase={}bodyz%Total Latency for this API call is {}statuscoder) exceptionZcurrent_attemptZtotal_time_elapsedrMz7Retry attempt: {}, Http Status Code: {}, Error Code: {})timerr3typerr:rgetattrrQrPrr6r5rr Exception should_retryrMdo_sleepr )r rHrIrJrZZ max_attemptZlog_infoattemptZ start_timeZ&_should_record_body_position_for_retryZ_is_body_retryabler3ZcheckerZ base_clientZ body_positionresponseZ time_elapsedeZ status_codeZ error_coder"r"r#rKs\         z6ExponentialBackoffRetryStrategyBase.make_retrying_callcCstd)Nz Subclasses should implement this)NotImplementedError)r r\rUr"r"r#r[Psz,ExponentialBackoffRetryStrategyBase.do_sleepcCs ||_dSN)rM)r rMr"r"r#add_circuit_breaker_callbackSs z@ExponentialBackoffRetryStrategyBase.add_circuit_breaker_callbackN)rCrDrErFr$rKr[rar"r"r"r#rLs @ rLcs,eZdZdZ dfdd ZddZZS)r;a4 A retry strategy which does exponential backoff with decorrelated jitter. Times used are in seconds and the strategy can be described as: .. code-block:: none min(base_sleep_time_seconds * exponent_growth_factor ** (attempt) + random(0, 1), max_wait_seconds)) rc s(tt|j||||fi|||_dS)ah Creates a new instance of an exponential backoff with Decorrelated jitter retry strategy. :param int base_sleep_time_seconds: The base amount to sleep by, in seconds :param int exponent_growth_factor: The exponent part of our backoff. We will raise take this value and raising it to the power of attempts and then multiply this with base_sleep_time_seconds :param int max_wait_between_calls_seconds: The maximum time we will wait between calls, in seconds :param retry_checkers.RetryCheckerContainer checker_container: The checks to run to determine whether a failed call should be retried :param int decorrelated_jitter (optional): The amount of time in seconds (default 1) to be used as de-correlated jitter between subsequent retries. N)superr;r$r )r r4r5r6r7r r! __class__r"r#r$bs z>ExponentialBackOffWithDecorrelatedJitterRetryStrategy.__init__cCs(t|j|j|j||j}t|dSr`)rZ;get_exponential_backoff_with_decorrelated_jitter_sleep_timer4r5r6r rVsleepr r\rUZsleep_time_subsecondsr"r"r#r[|sz>ExponentialBackOffWithDecorrelatedJitterRetryStrategy.do_sleep)rrCrDrErFr$r[ __classcell__r"r"rcr#r;Ws  r;c(eZdZdZfddZddZZS)r=a A retry strategy which does exponential backoff and full jitter. Times used are in seconds and the strategy can be described as: .. code-block:: none random(0, min(base_sleep_time_seconds * exponent_growth_factor ** (attempt), max_wait_seconds)) c "tt|j||||fi|dS)a Creates a new instance of an exponential backoff with full jitter retry strategy. :param int base_sleep_time_seconds: The base amount to sleep by, in seconds :param int exponent_growth_factor: The exponent part of our backoff. We will raise take this value and raising it to the power of attempts and then multiply this with base_sleep_time_seconds :param int max_wait_between_calls_seconds: The maximum time we will wait between calls, in seconds :param retry_checkers.RetryCheckerContainer checker_container: The checks to run to determine whether a failed call should be retried N)rbr=r$rNrcr"r#r$   z6ExponentialBackoffWithFullJitterRetryStrategy.__init__cC$t|j|j|j|}t|dSr`)r3get_exponential_backoff_with_full_jitter_sleep_timer4r5r6rVrerfr"r"r#r[sz6ExponentialBackoffWithFullJitterRetryStrategy.do_sleeprgr"r"rcr#r=s r=cri)r?a A retry strategy which does exponential backoff and equal jitter. Times used are in seconds and the strategy can be described as: .. code-block:: none exponential_backoff_sleep = min(base_sleep_time_seconds * exponent_growth_factor ** (attempt), max_wait_seconds) sleep_with_jitter = (exponential_backoff_sleep / 2) + random(0, exponential_backoff_sleep / 2) c rj)a Creates a new instance of an exponential backoff with equal jitter retry strategy. :param int base_sleep_time_seconds: The base amount to sleep by, in seconds :param int exponent_growth_factor: The exponent part of our backoff. We will raise take this value and raising it to the power of attemps and then multiply this with base_sleep_time_seconds :param int max_wait_between_calls_seconds: The maximum time we will wait between calls, in seconds :param retry_checkers.RetryCheckerContainer checker_container: The checks to run to determine whether a failed call should be retried N)rbr?r$rNrcr"r#r$rkz7ExponentialBackoffWithEqualJitterRetryStrategy.__init__cCrlr`)r4get_exponential_backoff_with_equal_jitter_sleep_timer4r5r6rVrer r\rUZsleep_time_secondsr"r"r#r[sz7ExponentialBackoffWithEqualJitterRetryStrategy.do_sleeprgr"r"rcr#r?s r?cri)rAa$ A retry strategy that does exponential backoff and full jitter for most retries, but uses exponential backoff with equal jitter for throttles. This provides a reasonable distribution of retry times for most retryable error cases, but for throttles guarantees some sleep time c rj)a Creates a new instance of the retry strategy. :param int base_sleep_time_seconds: The base amount to sleep by, in seconds :param int exponent_growth_factor: The exponent part of our backoff. We will raise take this value and raising it to the power of attemps and then multiply this with base_sleep_time_seconds :param int max_wait_between_calls_seconds: The maximum time we will wait between calls, in seconds :param retry_checkers.RetryCheckerContainer checker_container: The checks to run to determine whether a failed call should be retried N)rbrAr$rNrcr"r#r$rkzGExponentialBackoffWithFullJitterEqualForThrottlesRetryStrategy.__init__cCsht|tr"|jdkrt|j|j|j|}nt|j|j|j|}n t|j|j|j|}t |dS)Ni) isinstancerrSrrnr4r5r6rmrVreror"r"r#r[s  zGExponentialBackoffWithFullJitterEqualForThrottlesRetryStrategy.do_sleeprgr"r"rcr#rAs rA) exceptionsrrrZoci.utilrrr rVr<r>r@rrobjectr r8rLr;r=r?rAr"r"r"r#s$    Jf+%%