Hi Kathees,

The underlying implementation of pool can have a significant impact on the
performance. See the architecture tread "bench marks for object pool
implementations"

What is reason for going for a custom implementation here? Since this is a
custom implementation it might be a good idea to do a micro-bench mark of
the pool and compare its performance with other implementations.

You can also profile the application to find out where the contentions etc
are and then improve the performance (by narrowing down lock scopes etc)

regards

Malith

On Wed, Nov 30, 2016 at 3:26 PM, Malaka Silva <mal...@wso2.com> wrote:

> Seems like a significant improvement.
>
> I guess pool exists per node in cluster?
>
> Once completed please arrange a code review.
>
> On Wed, Nov 30, 2016 at 12:27 PM, Kathees Rajendram <kath...@wso2.com>
> wrote:
>
>> Hi,
>>
>> I am working on connection pool implementation for Kafka [1] and JMS [2]
>> connectors. In the existing connectors, connections are created and
>> closed per message which gives the performance issue. I have improved the
>> Kafka connector with configurable connection pool parameter. When
>> publishing the messages, the maximum pool size parameter value can be
>> changed in the connector configuration. Now the implementation with
>> connection pool gives better performance.
>>
>> The following results are with connection pool implementation.
>> Average throughput -  4980/s
>>
>>
>> *Threads & Pool Size* *Messages per a Thread* *No of Messages* *A
>> Message Size(byte)*
>> *Throughput (/s)*
>> 10
>> 500 5,000 106 3728
>> 20 500 10,000 106 4757
>> 100
>> 2500 250,000 106 5283
>> 100 10000 1,000,000 106 4920
>>
>>
>> The following results were without connection pool implementation.
>> Average throughput -  2767/s
>>
>> *Threads* *Messages per a Thread* *No of Messages* *A Message Size(byte)*
>> *Throughput (/s)*
>> 10 500 5,000 106 1590
>> 20 500 10,000 106 2251
>> 100 250 25,000 106 2940
>> 1,000 1500 1,500,000 106 2780
>> In my git repo [3], I have committed the connection pool implementation
>> for Kafka connector. Please give your suggestion.
>>
>> [1] - https://wso2.org/jira/browse/ESBCONNECT-122
>> [2] - https://wso2.org/jira/browse/ESBCONNECT-142
>> [3] - https://github.com/RKathees/esb-connector-kafka/blob/master/
>> src/main/java/org/wso2/carbon/connector/KafkaConnectionPoolManager.java
>>
>> Thanks,
>> Kathees
>> --
>> Kathees
>> Software Engineer,
>> email: kath...@wso2.com
>> mobile: +94772596173 <077%20259%206173>
>>
>
>
>
> --
>
> Best Regards,
>
> Malaka Silva
> Senior Technical Lead
> M: +94 777 219 791 <+94%2077%20721%209791>
> Tel : 94 11 214 5345
> Fax :94 11 2145300
> Skype : malaka.sampath.silva
> LinkedIn : http://www.linkedin.com/pub/malaka-silva/6/33/77
> Blog : http://mrmalakasilva.blogspot.com/
>
> WSO2, Inc.
> lean . enterprise . middleware
> https://wso2.com/signature
> http://www.wso2.com/about/team/malaka-silva/
> <http://wso2.com/about/team/malaka-silva/>
> https://store.wso2.com/store/
>
> Don't make Trees rare, we should keep them with care
>
> _______________________________________________
> Architecture mailing list
> Architecture@wso2.org
> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
>
>


-- 
Malith Jayasinghe

WSO2, Inc. (http://wso2.com)
Email   : mali...@wso2.com
Mobile : 0770704040
Lean . Enterprise . Middleware
_______________________________________________
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture

Reply via email to