Ifat Afek created BEAM-14438:
--------------------------------
Summary: Beam slowness compared to flink-native
Key: BEAM-14438
URL: https://issues.apache.org/jira/browse/BEAM-14438
Project: Beam
Issue Type: Bug
Components: runner-flink
Affects Versions: 2.38.0
Reporter: Ifat Afek
I tried to compare a very simple beam pipeline with an equivalent flink-native
pipeline. Both pipelines read strings from one kafka topic and write them to
another topic. I ran each pipeline separately on a single task manager with a
single slot and parallelism 1.
Flink native runs 5 times faster than beam: 150,000 strings per second in flink
comparing to 30,000 in beam.
When using Avro and schema registry the difference is even more significant -
flink native runs 30-80 times faster than beam.
Attached is the java code of both string-to-string pipelines.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
