kennknowles commented on issue #21569: URL: https://github.com/apache/beam/issues/21569#issuecomment-1400744712
I know that Flink has a highly tuned Kafka connector. And we do expect _some_ overhead since Beam will add a couple layers of virtual method calls. If there are inefficiencies in serialization that will be a serious overhead. The other thing I should say is of course in streaming we care primarily about cost and maximum scale so limiting everything to 1 task manager etc is not necessarily measuring what you care about. It is still useful for finding overhead, for sure. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
