Package org.apache.storm.kafka.spout
Class KafkaSpoutRetryExponentialBackoff
- java.lang.Object
-
- org.apache.storm.kafka.spout.KafkaSpoutRetryExponentialBackoff
-
- All Implemented Interfaces:
Serializable,KafkaSpoutRetryService
public class KafkaSpoutRetryExponentialBackoff extends Object implements KafkaSpoutRetryService
Implementation ofKafkaSpoutRetryServiceusing the exponential backoff formula. The time of the nextRetry is set as follows: nextRetry = failCount == 1 ? currentTime + initialDelay : currentTime + delayPeriod*2^(failCount-1) where failCount = 1, 2, 3, ... nextRetry = Min(nextRetry, currentTime + maxDelay)- See Also:
- Serialized Form
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description static classKafkaSpoutRetryExponentialBackoff.TimeInterval
-
Constructor Summary
Constructors Constructor Description KafkaSpoutRetryExponentialBackoff(KafkaSpoutRetryExponentialBackoff.TimeInterval initialDelay, KafkaSpoutRetryExponentialBackoff.TimeInterval delayPeriod, int maxRetries, KafkaSpoutRetryExponentialBackoff.TimeInterval maxDelay)The time stamp of the next retry is scheduled according to the exponential backoff formula (geometric progression): nextRetry = failCount == 1 ? currentTime + initialDelay : currentTime + delayPeriod^(failCount-1), where failCount = 1, 2, 3, ...
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description Map<org.apache.kafka.common.TopicPartition,Long>earliestRetriableOffsets()Gets the earliest retriable offsets.KafkaSpoutMessageIdgetMessageId(org.apache.kafka.common.TopicPartition tp, long offset)Gets theKafkaSpoutMessageIdfor the record on the given topic partition and offset.booleanisReady(KafkaSpoutMessageId msgId)Checks if a specific failedKafkaSpoutMessageIdis ready to be retried, i.e is scheduled and has retry time that is less than current time.booleanisScheduled(KafkaSpoutMessageId msgId)Checks if a specific failedKafkaSpoutMessageIdis scheduled to be retried.intreadyMessageCount()Get the number of messages ready for retry.booleanremove(KafkaSpoutMessageId msgId)Removes a message from the list of messages scheduled for retrial.booleanretainAll(Collection<org.apache.kafka.common.TopicPartition> topicPartitions)Retains all the messages whoseTopicPartitionbelongs to the specifiedCollection<TopicPartition>.booleanschedule(KafkaSpoutMessageId msgId)Schedules thisKafkaSpoutMessageIdif not yet scheduled, or updates retry time if it has already been scheduled.StringtoString()
-
-
-
Constructor Detail
-
KafkaSpoutRetryExponentialBackoff
public KafkaSpoutRetryExponentialBackoff(KafkaSpoutRetryExponentialBackoff.TimeInterval initialDelay, KafkaSpoutRetryExponentialBackoff.TimeInterval delayPeriod, int maxRetries, KafkaSpoutRetryExponentialBackoff.TimeInterval maxDelay)
The time stamp of the next retry is scheduled according to the exponential backoff formula (geometric progression): nextRetry = failCount == 1 ? currentTime + initialDelay : currentTime + delayPeriod^(failCount-1), where failCount = 1, 2, 3, ... nextRetry = Min(nextRetry, currentTime + maxDelay). By specifying a value for maxRetries lower than Integer.MAX_VALUE, the user decides to sacrifice guarantee of delivery for the previous polled records in favor of processing more records.- Parameters:
initialDelay- initial delay of the first retrydelayPeriod- the time interval that is the ratio of the exponential backoff formula (geometric progression)maxRetries- maximum number of times a tuple is retried before being acked and scheduled for commitmaxDelay- maximum amount of time waiting before retrying
-
-
Method Detail
-
earliestRetriableOffsets
public Map<org.apache.kafka.common.TopicPartition,Long> earliestRetriableOffsets()
Description copied from interface:KafkaSpoutRetryServiceGets the earliest retriable offsets.- Specified by:
earliestRetriableOffsetsin interfaceKafkaSpoutRetryService- Returns:
- The earliest retriable offset for each TopicPartition that has offsets ready to be retried, i.e. for which a tuple has failed and has retry time less than current time.
-
isReady
public boolean isReady(KafkaSpoutMessageId msgId)
Description copied from interface:KafkaSpoutRetryServiceChecks if a specific failedKafkaSpoutMessageIdis ready to be retried, i.e is scheduled and has retry time that is less than current time.- Specified by:
isReadyin interfaceKafkaSpoutRetryService- Parameters:
msgId- message to check for readiness- Returns:
- true if message is ready to be retried, false otherwise
-
isScheduled
public boolean isScheduled(KafkaSpoutMessageId msgId)
Description copied from interface:KafkaSpoutRetryServiceChecks if a specific failedKafkaSpoutMessageIdis scheduled to be retried. The message may or may not be ready to be retried yet.- Specified by:
isScheduledin interfaceKafkaSpoutRetryService- Parameters:
msgId- message to check for scheduling status- Returns:
- true if the message is scheduled to be retried, regardless of being or not ready to be retried. Returns false is this message is not scheduled for retrial
-
remove
public boolean remove(KafkaSpoutMessageId msgId)
Description copied from interface:KafkaSpoutRetryServiceRemoves a message from the list of messages scheduled for retrial.- Specified by:
removein interfaceKafkaSpoutRetryService- Parameters:
msgId- message to remove from retrial- Returns:
- true if the message was scheduled for retrial, false otherwise
-
retainAll
public boolean retainAll(Collection<org.apache.kafka.common.TopicPartition> topicPartitions)
Description copied from interface:KafkaSpoutRetryServiceRetains all the messages whoseTopicPartitionbelongs to the specifiedCollection<TopicPartition>. All messages that come from aTopicPartitionNOT existing in the collection will be removed. This method is useful to cleanup state following partition rebalance.- Specified by:
retainAllin interfaceKafkaSpoutRetryService- Parameters:
topicPartitions- Collection ofTopicPartitionfor which to keep messages- Returns:
- true if at least one message was removed, false otherwise
-
schedule
public boolean schedule(KafkaSpoutMessageId msgId)
Description copied from interface:KafkaSpoutRetryServiceSchedules thisKafkaSpoutMessageIdif not yet scheduled, or updates retry time if it has already been scheduled. It may also indicate that the message should not be retried, in which case the message will not be scheduled.- Specified by:
schedulein interfaceKafkaSpoutRetryService- Parameters:
msgId- message to schedule for retrial- Returns:
- true if the message will be retried, false otherwise
-
readyMessageCount
public int readyMessageCount()
Description copied from interface:KafkaSpoutRetryServiceGet the number of messages ready for retry.- Specified by:
readyMessageCountin interfaceKafkaSpoutRetryService- Returns:
- The number of messages that are ready for retry
-
getMessageId
public KafkaSpoutMessageId getMessageId(org.apache.kafka.common.TopicPartition tp, long offset)
Description copied from interface:KafkaSpoutRetryServiceGets theKafkaSpoutMessageIdfor the record on the given topic partition and offset.- Specified by:
getMessageIdin interfaceKafkaSpoutRetryService- Parameters:
tp- The topic partition of the recordoffset- The offset of the record- Returns:
- The id the record was scheduled for retry with,
or a new
KafkaSpoutMessageIdif the record was not scheduled for retry.
-
-