[
https://issues.apache.org/jira/browse/METRON-2157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16864124#comment-16864124
]
Simon Elliston Ball commented on METRON-2157:
---------------------------------------------
My one worry about it would be that it's a potentially more complex abstraction
to tune, given the metron focus on high throughput. The beam programming model
might fit our current use cases, but is a least common denominator. That said
there has been a lot of work recently to abstract storm out, so beam could be a
viable engine to run our business logic on. The challenge would then be to
figure out the optimal runtime underneath that. I suspect this would benefit
from a discuss thread on the mail lists.
> To better abstract the streaming infrastructure, use Apache Beam.
> -----------------------------------------------------------------
>
> Key: METRON-2157
> URL: https://issues.apache.org/jira/browse/METRON-2157
> Project: Metron
> Issue Type: Improvement
> Reporter: Phillip Henry
> Priority: Major
>
> Apache Beam is "[a]n advanced unified programming model" that can "Implement
> batch and streaming data processing jobs that run on any execution engine"
> (according to their [docs|[https://beam.apache.org/]).
> If Metron were to use such an abstraction, there could be greater portability
> between different offerings of streaming infrastructure (Spark, Flink etc).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
