damccorm opened a new issue, #21569: URL: https://github.com/apache/beam/issues/21569
I tried to compare a very simple beam pipeline with an equivalent flink-native pipeline. Both pipelines read strings from one kafka topic and write them to another topic. I ran each pipeline separately on a single task manager with a single slot and parallelism 1. Flink native runs 5 times faster than beam: 150,000 strings per second in flink comparing to 30,000 in beam. When using Avro and schema registry the difference is even more significant - flink native runs 30-80 times faster than beam. Attached is the java code of both string-to-string pipelines. Imported from Jira [BEAM-14438](https://issues.apache.org/jira/browse/BEAM-14438). Original Jira may contain additional context. Reported by: iafek. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
