[
https://issues.apache.org/jira/browse/STORM-1757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15281795#comment-15281795
]
Robert Joseph Evans commented on STORM-1757:
--------------------------------------------
I have a customer that is potentially interested in this too, but I need to
come up with a plan/LOE to give them some advice.
For the most part it seems like we should be able to build something fairly
quickly, but like you said things don't seem too stable over on the BEAM side.
I am going to probe a bit more on the mailing lists to try and understand some
of the details in Windowing/Triggering, and side data.
There are a few primitive operations that we need to implement a
translation/runner for.
GroupByKey - Groups by key and window
CreatePCollectionView - for side input, not sure how/if this works with
streaming
Read.Bounded - only for batch (which we probably can support)
Read.Unbounded
ParDo.Bound
ParDo.BoundMulti
FlattenPCollectionList
Window.Bound
I am still working things out, but the main difference between what we have
done so far and what we have right now with trident windowing/checkpointing is
around the triggers and the watermarks. I don't know the internals of the
windowing/checkpointing work in storm well enough to really say that if we had
a trident implementation it would work, but I don't think so. I still have a
bit more I need to understand before I can come up with a final design.
> Apache Beam Runner for Storm
> ----------------------------
>
> Key: STORM-1757
> URL: https://issues.apache.org/jira/browse/STORM-1757
> Project: Apache Storm
> Issue Type: Brainstorming
> Reporter: P. Taylor Goetz
> Priority: Minor
>
> This is a call for interested parties to collaborate on an Apache Beam [1]
> runner for Storm, and express their thoughts and opinions.
> Given the addition of the Windowing API to Apache Storm, we should be able to
> map naturally to the Beam API. If not, it may be indicative of shortcomings
> of the Storm API that should be addressed.
> [1] http://beam.incubator.apache.org
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
