Seems like a significant improvement. I guess pool exists per node in cluster?
Once completed please arrange a code review. On Wed, Nov 30, 2016 at 12:27 PM, Kathees Rajendram <[email protected]> wrote: > Hi, > > I am working on connection pool implementation for Kafka [1] and JMS [2] > connectors. In the existing connectors, connections are created and > closed per message which gives the performance issue. I have improved the > Kafka connector with configurable connection pool parameter. When > publishing the messages, the maximum pool size parameter value can be > changed in the connector configuration. Now the implementation with > connection pool gives better performance. > > The following results are with connection pool implementation. > Average throughput - 4980/s > > > *Threads & Pool Size* *Messages per a Thread* *No of Messages* *A Message > Size(byte)* > *Throughput (/s)* > 10 > 500 5,000 106 3728 > 20 500 10,000 106 4757 > 100 > 2500 250,000 106 5283 > 100 10000 1,000,000 106 4920 > > > The following results were without connection pool implementation. > Average throughput - 2767/s > > *Threads* *Messages per a Thread* *No of Messages* *A Message Size(byte)* > *Throughput (/s)* > 10 500 5,000 106 1590 > 20 500 10,000 106 2251 > 100 250 25,000 106 2940 > 1,000 1500 1,500,000 106 2780 > In my git repo [3], I have committed the connection pool implementation > for Kafka connector. Please give your suggestion. > > [1] - https://wso2.org/jira/browse/ESBCONNECT-122 > [2] - https://wso2.org/jira/browse/ESBCONNECT-142 > [3] - https://github.com/RKathees/esb-connector-kafka/blob/ > master/src/main/java/org/wso2/carbon/connector/KafkaConnectionPoolManager. > java > > Thanks, > Kathees > -- > Kathees > Software Engineer, > email: [email protected] > mobile: +94772596173 <077%20259%206173> > -- Best Regards, Malaka Silva Senior Technical Lead M: +94 777 219 791 Tel : 94 11 214 5345 Fax :94 11 2145300 Skype : malaka.sampath.silva LinkedIn : http://www.linkedin.com/pub/malaka-silva/6/33/77 Blog : http://mrmalakasilva.blogspot.com/ WSO2, Inc. lean . enterprise . middleware https://wso2.com/signature http://www.wso2.com/about/team/malaka-silva/ <http://wso2.com/about/team/malaka-silva/> https://store.wso2.com/store/ Don't make Trees rare, we should keep them with care
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
