Hi Malaka, Yes. Once I compare this performance with other implementations and will arrange a code review.
@Malith I will compare the performance of this custom connection pool implementation with other implementations as it is mentioned in the mail thread [1] that Stormpot throughput is highest, will check the possibility to add in the connector and analyze performance results. [1] - "[Architecture] Benchmarks for Object Pool Implementations" Thanks, Kathees On Thu, Dec 1, 2016 at 9:29 AM, Malith Jayasinghe <[email protected]> wrote: > Hi Kathees, > > The underlying implementation of pool can have a significant impact on the > performance. See the architecture tread "bench marks for object pool > implementations" > > What is reason for going for a custom implementation here? Since this is a > custom implementation it might be a good idea to do a micro-bench mark of > the pool and compare its performance with other implementations. > > You can also profile the application to find out where the contentions etc > are and then improve the performance (by narrowing down lock scopes etc) > > regards > > Malith > > On Wed, Nov 30, 2016 at 3:26 PM, Malaka Silva <[email protected]> wrote: > >> Seems like a significant improvement. >> >> I guess pool exists per node in cluster? >> >> Once completed please arrange a code review. >> >> On Wed, Nov 30, 2016 at 12:27 PM, Kathees Rajendram <[email protected]> >> wrote: >> >>> Hi, >>> >>> I am working on connection pool implementation for Kafka [1] and JMS [2] >>> connectors. In the existing connectors, connections are created and >>> closed per message which gives the performance issue. I have improved the >>> Kafka connector with configurable connection pool parameter. When >>> publishing the messages, the maximum pool size parameter value can be >>> changed in the connector configuration. Now the implementation with >>> connection pool gives better performance. >>> >>> The following results are with connection pool implementation. >>> Average throughput - 4980/s >>> >>> >>> *Threads & Pool Size* *Messages per a Thread* *No of Messages* *A >>> Message Size(byte)* >>> *Throughput (/s)* >>> 10 >>> 500 5,000 106 3728 >>> 20 500 10,000 106 4757 >>> 100 >>> 2500 250,000 106 5283 >>> 100 10000 1,000,000 106 4920 >>> >>> >>> The following results were without connection pool implementation. >>> Average throughput - 2767/s >>> >>> *Threads* *Messages per a Thread* *No of Messages* *A Message >>> Size(byte)* >>> *Throughput (/s)* >>> 10 500 5,000 106 1590 >>> 20 500 10,000 106 2251 >>> 100 250 25,000 106 2940 >>> 1,000 1500 1,500,000 106 2780 >>> In my git repo [3], I have committed the connection pool implementation >>> for Kafka connector. Please give your suggestion. >>> >>> [1] - https://wso2.org/jira/browse/ESBCONNECT-122 >>> [2] - https://wso2.org/jira/browse/ESBCONNECT-142 >>> [3] - https://github.com/RKathees/esb-connector-kafka/blob/master/ >>> src/main/java/org/wso2/carbon/connector/KafkaConnectionPoolManager.java >>> >>> Thanks, >>> Kathees >>> -- >>> Kathees >>> Software Engineer, >>> email: [email protected] >>> mobile: +94772596173 <077%20259%206173> >>> >> >> >> >> -- >> >> Best Regards, >> >> Malaka Silva >> Senior Technical Lead >> M: +94 777 219 791 <+94%2077%20721%209791> >> Tel : 94 11 214 5345 >> Fax :94 11 2145300 >> Skype : malaka.sampath.silva >> LinkedIn : http://www.linkedin.com/pub/malaka-silva/6/33/77 >> Blog : http://mrmalakasilva.blogspot.com/ >> >> WSO2, Inc. >> lean . enterprise . middleware >> https://wso2.com/signature >> http://www.wso2.com/about/team/malaka-silva/ >> <http://wso2.com/about/team/malaka-silva/> >> https://store.wso2.com/store/ >> >> Don't make Trees rare, we should keep them with care >> >> _______________________________________________ >> Architecture mailing list >> [email protected] >> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >> >> > > > -- > Malith Jayasinghe > > WSO2, Inc. (http://wso2.com) > Email : [email protected] > Mobile : 0770704040 > Lean . Enterprise . Middleware > -- Kathees Software Engineer, email: [email protected] mobile: +94772596173
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
