[
https://issues.apache.org/jira/browse/STORM-1167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14990107#comment-14990107
]
ASF GitHub Bot commented on STORM-1167:
---------------------------------------
Github user haohui commented on the pull request:
https://github.com/apache/storm/pull/855#issuecomment-153820936
The high level API looks good to me overall. It maintains a view of all the
events in the window which is a powerful concept.
I have several questions on how this PR can help implement two common use
cases.
(1) Aggregation (e.g., min / max) over a sliding window
(2) Stream joins over a large amount of data
The abstractions of views in memory are insufficient because for an
efficient algorithm for (1) does not need every single events in the window,
and (2) the events in the window need to be spilled to secondary storage. To me
it seems that it still requires writing a lot of custom code. The issue might
be mitigated to add flexibility on whether and where to keep the events in the
window in the API.
> Add sliding & tumbling window support for core storm
> ----------------------------------------------------
>
> Key: STORM-1167
> URL: https://issues.apache.org/jira/browse/STORM-1167
> Project: Apache Storm
> Issue Type: Improvement
> Reporter: Arun Mahadevan
> Assignee: Arun Mahadevan
>
> Currently, topologies that needs windowing support requires writing custom
> logic inside bolts making it tedious to handle the windowing and acking logic
> with custom logic.
> We can add framework level support to core storm bolts to process tuples in a
> time or a count based window. Sliding and tumbling windows can be supported.
> Later this can be extended to trident apis as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
