HeartSaVioR commented on issue #22138: [SPARK-25151][SS] Apply Apache Commons Pool to KafkaDataConsumer URL: https://github.com/apache/spark/pull/22138#issuecomment-468700176 Yes, no-op sink is placed to get rid of unnecessary overhead. I have tool for test-data so the query needs to do some work (json parsing and a bit more). Right it hits the same topic partition multiple times, and that's the basic case we expect cache to cover. We could measure performance for non-basic cases like adding partitions if we want. Do you have some scenarios to test in mind?
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
