gaborgsomogyi commented on issue #25853: [SPARK-21869][SS] Apply Apache Commons Pool to Kafka producer URL: https://github.com/apache/spark/pull/25853#issuecomment-548511277 First of all thanks for your efforts @HeartSaVioR I've just analyzed through the application what you've tested with. All I can say it's super useful and starred it for later use (haven't found logical issues in the app). `processedRowsPerSecond` median fluctuation is almost the same just like in my single node case. The almost 3x better performance looks interesting though. Just for the sake of my understanding: * How many nodes are in your Spark cluster? * Are the Kafka nodes running in the same cluster? * How many brokers contains the Kafka setup?
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
