Package org.apache.storm.topology.base
Class BaseStatefulWindowedBolt<T extends State>
- java.lang.Object
-
- org.apache.storm.topology.base.BaseWindowedBolt
-
- org.apache.storm.topology.base.BaseStatefulWindowedBolt<T>
-
- All Implemented Interfaces:
Serializable,IComponent,IStatefulComponent<T>,IStatefulWindowedBolt<T>,IWindowedBolt
public abstract class BaseStatefulWindowedBolt<T extends State> extends BaseWindowedBolt implements IStatefulWindowedBolt<T>
- See Also:
- Serialized Form
-
-
Nested Class Summary
-
Nested classes/interfaces inherited from class org.apache.storm.topology.base.BaseWindowedBolt
BaseWindowedBolt.Count, BaseWindowedBolt.Duration
-
-
Field Summary
-
Fields inherited from class org.apache.storm.topology.base.BaseWindowedBolt
timestampExtractor, windowConfiguration
-
-
Constructor Summary
Constructors Constructor Description BaseStatefulWindowedBolt()
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description booleanisPersistent()If the stateful windowed bolt should have its windows persisted in state and maintain a subset of events in memory.longmaxEventsInMemory()The maximum number of window events to keep in memory.voidpreCommit(long txid)This is a hook for the component to perform some actions just before the framework commits its state.voidprePrepare(long txid)This is a hook for the component to perform some actions just before the framework prepares its state.voidpreRollback()This is a hook for the component to perform some actions just before the framework rolls back the prepared state.BaseStatefulWindowedBolt<T>withLag(BaseWindowedBolt.Duration duration)Specify the maximum time lag of the tuple timestamp in milliseconds.BaseStatefulWindowedBolt<T>withLateTupleStream(String streamName)Specify a stream id on which late tuples are going to be emitted.BaseStatefulWindowedBolt<T>withMaxEventsInMemory(long maxEventsInMemory)The maximum number of window events to keep in memory.BaseStatefulWindowedBolt<T>withMessageIdField(String fieldName)Specify the name of the field in the tuple that holds the message id.BaseStatefulWindowedBolt<T>withPersistence()If set, the stateful windowed bolt would use the backend state for window persistence and only keep a sub-set of events in memory as specified bywithMaxEventsInMemory(long).BaseStatefulWindowedBolt<T>withTimestampExtractor(TimestampExtractor timestampExtractor)Specify the timestamp extractor implementation.BaseStatefulWindowedBolt<T>withTimestampField(String fieldName)Specify a field in the tuple that represents the timestamp as a long value.BaseStatefulWindowedBolt<T>withTumblingWindow(BaseWindowedBolt.Count count)A count based tumbling window.BaseStatefulWindowedBolt<T>withTumblingWindow(BaseWindowedBolt.Duration duration)A time duration based tumbling window.BaseStatefulWindowedBolt<T>withWatermarkInterval(BaseWindowedBolt.Duration interval)Specify the watermark event generation interval.BaseStatefulWindowedBolt<T>withWindow(BaseWindowedBolt.Count windowLength)A tuple count based window that slides with every incoming tuple.BaseStatefulWindowedBolt<T>withWindow(BaseWindowedBolt.Count windowLength, BaseWindowedBolt.Count slidingInterval)Tuple count based sliding window configuration.BaseStatefulWindowedBolt<T>withWindow(BaseWindowedBolt.Count windowLength, BaseWindowedBolt.Duration slidingInterval)Tuple count and time duration based sliding window configuration.BaseStatefulWindowedBolt<T>withWindow(BaseWindowedBolt.Duration windowLength)A time duration based window that slides with every incoming tuple.BaseStatefulWindowedBolt<T>withWindow(BaseWindowedBolt.Duration windowLength, BaseWindowedBolt.Count slidingInterval)Time duration and count based sliding window configuration.BaseStatefulWindowedBolt<T>withWindow(BaseWindowedBolt.Duration windowLength, BaseWindowedBolt.Duration slidingInterval)Time duration based sliding window configuration.-
Methods inherited from class org.apache.storm.topology.base.BaseWindowedBolt
cleanup, declareOutputFields, getComponentConfiguration, getTimestampExtractor, prepare
-
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
-
Methods inherited from interface org.apache.storm.topology.IComponent
declareOutputFields, getComponentConfiguration
-
Methods inherited from interface org.apache.storm.topology.IStatefulComponent
initState
-
Methods inherited from interface org.apache.storm.topology.IWindowedBolt
cleanup, execute, getTimestampExtractor, prepare
-
-
-
-
Method Detail
-
withWindow
public BaseStatefulWindowedBolt<T> withWindow(BaseWindowedBolt.Count windowLength, BaseWindowedBolt.Count slidingInterval)
Tuple count based sliding window configuration.- Overrides:
withWindowin classBaseWindowedBolt- Parameters:
windowLength- the number of tuples in the windowslidingInterval- the number of tuples after which the window slides
-
withWindow
public BaseStatefulWindowedBolt<T> withWindow(BaseWindowedBolt.Count windowLength, BaseWindowedBolt.Duration slidingInterval)
Tuple count and time duration based sliding window configuration.- Overrides:
withWindowin classBaseWindowedBolt- Parameters:
windowLength- the number of tuples in the windowslidingInterval- the time duration after which the window slides
-
withWindow
public BaseStatefulWindowedBolt<T> withWindow(BaseWindowedBolt.Duration windowLength, BaseWindowedBolt.Count slidingInterval)
Time duration and count based sliding window configuration.- Overrides:
withWindowin classBaseWindowedBolt- Parameters:
windowLength- the time duration of the windowslidingInterval- the number of tuples after which the window slides
-
withWindow
public BaseStatefulWindowedBolt<T> withWindow(BaseWindowedBolt.Duration windowLength, BaseWindowedBolt.Duration slidingInterval)
Time duration based sliding window configuration.- Overrides:
withWindowin classBaseWindowedBolt- Parameters:
windowLength- the time duration of the windowslidingInterval- the time duration after which the window slides
-
withWindow
public BaseStatefulWindowedBolt<T> withWindow(BaseWindowedBolt.Count windowLength)
A tuple count based window that slides with every incoming tuple.- Overrides:
withWindowin classBaseWindowedBolt- Parameters:
windowLength- the number of tuples in the window
-
withWindow
public BaseStatefulWindowedBolt<T> withWindow(BaseWindowedBolt.Duration windowLength)
A time duration based window that slides with every incoming tuple.- Overrides:
withWindowin classBaseWindowedBolt- Parameters:
windowLength- the time duration of the window
-
withTumblingWindow
public BaseStatefulWindowedBolt<T> withTumblingWindow(BaseWindowedBolt.Count count)
A count based tumbling window.- Overrides:
withTumblingWindowin classBaseWindowedBolt- Parameters:
count- the number of tuples after which the window tumbles
-
withTumblingWindow
public BaseStatefulWindowedBolt<T> withTumblingWindow(BaseWindowedBolt.Duration duration)
A time duration based tumbling window.- Overrides:
withTumblingWindowin classBaseWindowedBolt- Parameters:
duration- the time duration after which the window tumbles
-
withTimestampField
public BaseStatefulWindowedBolt<T> withTimestampField(String fieldName)
Specify a field in the tuple that represents the timestamp as a long value. If this field is not present in the incoming tuple, anIllegalArgumentExceptionwill be thrown. The field MUST contain a timestamp in milliseconds- Overrides:
withTimestampFieldin classBaseWindowedBolt- Parameters:
fieldName- the name of the field that contains the timestamp
-
withTimestampExtractor
public BaseStatefulWindowedBolt<T> withTimestampExtractor(TimestampExtractor timestampExtractor)
Specify the timestamp extractor implementation.- Overrides:
withTimestampExtractorin classBaseWindowedBolt- Parameters:
timestampExtractor- theTimestampExtractorimplementation
-
withLateTupleStream
public BaseStatefulWindowedBolt<T> withLateTupleStream(String streamName)
Specify a stream id on which late tuples are going to be emitted. They are going to be accessible via theWindowedBoltExecutor.LATE_TUPLE_FIELDfield. It must be defined on a per-component basis, and in conjunction with theBaseWindowedBolt.withTimestampField(java.lang.String), otherwiseIllegalArgumentExceptionwill be thrown.- Overrides:
withLateTupleStreamin classBaseWindowedBolt- Parameters:
streamName- the name of the stream used to emit late tuples on
-
withLag
public BaseStatefulWindowedBolt<T> withLag(BaseWindowedBolt.Duration duration)
Specify the maximum time lag of the tuple timestamp in milliseconds. It means that the tuple timestamps cannot be out of order by more than this amount.- Overrides:
withLagin classBaseWindowedBolt- Parameters:
duration- the max lag duration
-
withWatermarkInterval
public BaseStatefulWindowedBolt<T> withWatermarkInterval(BaseWindowedBolt.Duration interval)
Specify the watermark event generation interval. For tuple based timestamps, watermark events are used to track the progress of time- Overrides:
withWatermarkIntervalin classBaseWindowedBolt- Parameters:
interval- the interval at which watermark events are generated
-
withMessageIdField
public BaseStatefulWindowedBolt<T> withMessageIdField(String fieldName)
Specify the name of the field in the tuple that holds the message id. This is used to track the windowing boundaries and re-evaluating the windowing operation during recovery of IStatefulWindowedBolt- Parameters:
fieldName- the name of the field that contains the message id
-
withPersistence
public BaseStatefulWindowedBolt<T> withPersistence()
If set, the stateful windowed bolt would use the backend state for window persistence and only keep a sub-set of events in memory as specified bywithMaxEventsInMemory(long).
-
withMaxEventsInMemory
public BaseStatefulWindowedBolt<T> withMaxEventsInMemory(long maxEventsInMemory)
The maximum number of window events to keep in memory. This is meaningful only ifwithPersistence()is also set. As the number of events in memory grows close to the maximum, the events that are less likely to be used again are evicted and persisted. The default value for this is1,000,000.- Parameters:
maxEventsInMemory- the maximum number of window events to keep in memory
-
isPersistent
public boolean isPersistent()
Description copied from interface:IStatefulWindowedBoltIf the stateful windowed bolt should have its windows persisted in state and maintain a subset of events in memory.The default is to keep all the window events in memory.
- Specified by:
isPersistentin interfaceIStatefulWindowedBolt<T extends State>- Returns:
- true if the windows should be persisted
-
maxEventsInMemory
public long maxEventsInMemory()
Description copied from interface:IStatefulWindowedBoltThe maximum number of window events to keep in memory.- Specified by:
maxEventsInMemoryin interfaceIStatefulWindowedBolt<T extends State>
-
preCommit
public void preCommit(long txid)
Description copied from interface:IStatefulComponentThis is a hook for the component to perform some actions just before the framework commits its state.- Specified by:
preCommitin interfaceIStatefulComponent<T extends State>
-
prePrepare
public void prePrepare(long txid)
Description copied from interface:IStatefulComponentThis is a hook for the component to perform some actions just before the framework prepares its state.- Specified by:
prePreparein interfaceIStatefulComponent<T extends State>
-
preRollback
public void preRollback()
Description copied from interface:IStatefulComponentThis is a hook for the component to perform some actions just before the framework rolls back the prepared state.- Specified by:
preRollbackin interfaceIStatefulComponent<T extends State>
-
-