Hi, I am working on connection pool implementation for Kafka [1] and JMS [2] connectors. In the existing connectors, connections are created and closed per message which gives the performance issue. I have improved the Kafka connector with configurable connection pool parameter. When publishing the messages, the maximum pool size parameter value can be changed in the connector configuration. Now the implementation with connection pool gives better performance.
The following results are with connection pool implementation. Average throughput - 4980/s *Threads & Pool Size* *Messages per a Thread* *No of Messages* *A Message Size(byte)* *Throughput (/s)* 10 500 5,000 106 3728 20 500 10,000 106 4757 100 2500 250,000 106 5283 100 10000 1,000,000 106 4920 The following results were without connection pool implementation. Average throughput - 2767/s *Threads* *Messages per a Thread* *No of Messages* *A Message Size(byte)* *Throughput (/s)* 10 500 5,000 106 1590 20 500 10,000 106 2251 100 250 25,000 106 2940 1,000 1500 1,500,000 106 2780 In my git repo [3], I have committed the connection pool implementation for Kafka connector. Please give your suggestion. [1] - https://wso2.org/jira/browse/ESBCONNECT-122 [2] - https://wso2.org/jira/browse/ESBCONNECT-142 [3] - https://github.com/RKathees/esb-connector-kafka/blob/master/src/main/java/org/wso2/carbon/connector/KafkaConnectionPoolManager.java Thanks, Kathees -- Kathees Software Engineer, email: [email protected] mobile: +94772596173
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
