GitHub user arunmahadevan opened a pull request:
https://github.com/apache/storm/pull/939
[STORM-1175] State store for windowing operations
Added IStatefulBolt abstraction that can be implemented by bolts which
requires
its state to be checkpointed periodically. State implementations based on
key-value mapping store are added. There is a default in-memory based
implementation
and optional redis based implementation that provides state persistence. An
internal
CheckpointSpout periodically emits checkpoint tuples which flows through the
topology DAG to take a consistent snapshot of the state across all
components.
There is still pending work to capture the evaluated/expired state of the
tuples
in the Window and use it to prune duplicate window evaluations during
restart. This can
be built on top of the stateful bolt abstractions and will be done as part
of STORM-1176
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/arunmahadevan/storm windowing-state
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/storm/pull/939.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #939
----
commit f541f3f91aa2d287f56af4655d98ad0be4ba45dc
Author: Arun Mahadevan <[email protected]>
Date: 2015-11-30T11:06:10Z
[STORM-1175] State store for windowing operations
Added IStatefulBolt abstraction that can be implemented by bolts which
requires
its state to be checkpointed periodically. State implementations based on
key-value mapping store are added. There is a default in-memory based
implementation
and optional redis based implementation that provides state persistence. An
internal
CheckpointSpout periodically emits checkpoint tuples which flows through the
topology DAG to take a consistent snapshot of the state across all
components.
There is still pending work to capture the evaluated/expired state of the
tuples
in the Window and use it to prune duplicate window evaluations during
restart. This can
be built on top of the stateful bolt abstractions and will be done as part
of STORM-1176
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
