Soumabrata Chakraborty created BEAM-9440:
--------------------------------------------
Summary: Performance Issue with Beam
Key: BEAM-9440
URL: https://issues.apache.org/jira/browse/BEAM-9440
Project: Beam
Issue Type: Bug
Components: runner-spark
Reporter: Soumabrata Chakraborty
While doing a performance evaluation of Apache Beam with Spark Runner - I found
that even for a simple word count problem on a text file – Beam with Spark
runner was slower by a factor of 5 times as compared to Spark for a dataset as
small as 14 GB.
You will find more details on this evaluation here -
[https://github.com/soumabrata-chakraborty/spark-vs-beam/blob/master/README.md]
I also came across this analysis called _**Quantitative Impact Evaluation of an
Abstraction Layer for Data Stream Processing Systems_
([https://arxiv.org/pdf/1907.08302.pdf])
According to it, the observation was that for most scenarios the slowdown was
at least a factor of 3 with the worse case being a factor of 58!
While it is understood that an abstraction layer would come with some
performance cost - the current performance cost seems to be very high.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
