[
https://issues.apache.org/jira/browse/BEAM-9440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17057802#comment-17057802
]
Guenter Hesse commented on BEAM-9440:
-------------------------------------
When we started the work on this topic Apache Apex was a project still under
development. Unfortunately, shortly after submitting the final version of the
paper, we found out that the project was abandoned. So I totally agree with
you, Apex was/is not the best choice as a third system to compare, although it
is still listed as a supported system/runner:
[https://beam.apache.org/documentation/runners/capability-matrix/]
However, this page also shows Apache Gearpump and JStorm, both of which seem to
be retired by now, too. But that is probably a different topic, i.e., the
question when to remove abandoned projects from this matrix.
Regarding the Google Summer of Code project: that sounds very interesting to
me. In order to pass information to our students, could you please give me some
further details on how a collaboration would look like? You can send me an
email if that is convenient for you:
[[email protected]|mailto:[email protected]] Thanks!
> Performance Issues with Beam Runners compared with Native Systems
> -----------------------------------------------------------------
>
> Key: BEAM-9440
> URL: https://issues.apache.org/jira/browse/BEAM-9440
> Project: Beam
> Issue Type: Bug
> Components: runner-apex, runner-flink, runner-spark
> Reporter: Soumabrata Chakraborty
> Priority: Major
>
> While doing a performance evaluation of Apache Beam with Spark Runner - I
> found that even for a simple word count problem on a text file – Beam with
> Spark runner was slower by a factor of 5 times as compared to Spark for a
> dataset as small as 14 GB.
> You will find more details on this evaluation here -
> [https://github.com/soumabrata-chakraborty/spark-vs-beam/blob/master/README.md]
> I also came across this analysis called _**Quantitative Impact Evaluation of
> an Abstraction Layer for Data Stream Processing Systems_
> ([https://arxiv.org/pdf/1907.08302.pdf] /
> [https://ieeexplore.ieee.org/document/8884832])
> According to it, the observation was that for most scenarios the slowdown was
> at least a factor of 3 with the worse case being a factor of 58!
> While it is understood that an abstraction layer would come with some
> performance cost - the current performance cost seems to be very high.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
