Hi Dushan,

I am working on connection pool implementation for Kafka Connector. By next
week, I am going to start implementing the JMS connector.  I will let you
know while designing the connector.

Thanks,
Kathees

On Thu, Dec 1, 2016 at 12:34 PM, Dushan Abeyruwan <dus...@wso2.com> wrote:

> Hi Kathees,
>   Yes, there seems to be an obvious improvement. However, when comes to
> JMS connector,try to design that more generic, I guess that can be done
> with some customizations of what Hasitha has done.
>
> Anyway, please do let me know when JMS genetic connector design planned,
> we can run through the design of that connector.
>
> Have you started to write JMS connector separately ?
>
> Cheers,
> Dushan
>
> On Wed, Nov 30, 2016 at 10:39 PM, Kathees Rajendram <kath...@wso2.com>
> wrote:
>
>> Hi Malaka,
>>
>> Yes. Once I compare this performance with other implementations and will
>> arrange a code review.
>>
>> @Malith
>> I will compare the performance of this custom connection pool
>> implementation with other implementations as it is mentioned in the mail
>> thread [1] that Stormpot throughput is highest, will check the
>> possibility to add in the connector and analyze performance results.
>>
>> [1] -  "[Architecture] Benchmarks for Object Pool Implementations"
>>
>> Thanks,
>> Kathees
>>
>> On Thu, Dec 1, 2016 at 9:29 AM, Malith Jayasinghe <mali...@wso2.com>
>> wrote:
>>
>>> Hi Kathees,
>>>
>>> The underlying implementation of pool can have a significant impact on
>>> the performance. See the architecture tread "bench marks for object pool
>>> implementations"
>>>
>>> What is reason for going for a custom implementation here? Since this is
>>> a custom implementation it might be a good idea to do a micro-bench mark of
>>> the pool and compare its performance with other implementations.
>>>
>>> You can also profile the application to find out where the contentions
>>> etc are and then improve the performance (by narrowing down lock scopes
>>> etc)
>>>
>>> regards
>>>
>>> Malith
>>>
>>> On Wed, Nov 30, 2016 at 3:26 PM, Malaka Silva <mal...@wso2.com> wrote:
>>>
>>>> Seems like a significant improvement.
>>>>
>>>> I guess pool exists per node in cluster?
>>>>
>>>> Once completed please arrange a code review.
>>>>
>>>> On Wed, Nov 30, 2016 at 12:27 PM, Kathees Rajendram <kath...@wso2.com>
>>>> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> I am working on connection pool implementation for Kafka [1] and JMS
>>>>> [2] connectors. In the existing connectors, connections are created
>>>>> and closed per message which gives the performance issue. I have improved
>>>>> the Kafka connector with configurable connection pool parameter. When
>>>>> publishing the messages, the maximum pool size parameter value can be
>>>>> changed in the connector configuration. Now the implementation with
>>>>> connection pool gives better performance.
>>>>>
>>>>> The following results are with connection pool implementation.
>>>>> Average throughput -  4980/s
>>>>>
>>>>>
>>>>> *Threads & Pool Size* *Messages per a Thread* *No of Messages* *A
>>>>> Message Size(byte)*
>>>>> *Throughput (/s)*
>>>>> 10
>>>>> 500 5,000 106 3728
>>>>> 20 500 10,000 106 4757
>>>>> 100
>>>>> 2500 250,000 106 5283
>>>>> 100 10000 1,000,000 106 4920
>>>>>
>>>>>
>>>>> The following results were without connection pool implementation.
>>>>> Average throughput -  2767/s
>>>>>
>>>>> *Threads* *Messages per a Thread* *No of Messages* *A Message
>>>>> Size(byte)*
>>>>> *Throughput (/s)*
>>>>> 10 500 5,000 106 1590
>>>>> 20 500 10,000 106 2251
>>>>> 100 250 25,000 106 2940
>>>>> 1,000 1500 1,500,000 106 2780
>>>>> In my git repo [3], I have committed the connection pool
>>>>> implementation for Kafka connector. Please give your suggestion.
>>>>>
>>>>> [1] - https://wso2.org/jira/browse/ESBCONNECT-122
>>>>> [2] - https://wso2.org/jira/browse/ESBCONNECT-142
>>>>> [3] - https://github.com/RKathees/esb-connector-kafka/blob/master/
>>>>> src/main/java/org/wso2/carbon/connector/KafkaConnectionPoolM
>>>>> anager.java
>>>>>
>>>>> Thanks,
>>>>> Kathees
>>>>> --
>>>>> Kathees
>>>>> Software Engineer,
>>>>> email: kath...@wso2.com
>>>>> mobile: +94772596173 <077%20259%206173>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> Best Regards,
>>>>
>>>> Malaka Silva
>>>> Senior Technical Lead
>>>> M: +94 777 219 791 <+94%2077%20721%209791>
>>>> Tel : 94 11 214 5345
>>>> Fax :94 11 2145300
>>>> Skype : malaka.sampath.silva
>>>> LinkedIn : http://www.linkedin.com/pub/malaka-silva/6/33/77
>>>> Blog : http://mrmalakasilva.blogspot.com/
>>>>
>>>> WSO2, Inc.
>>>> lean . enterprise . middleware
>>>> https://wso2.com/signature
>>>> http://www.wso2.com/about/team/malaka-silva/
>>>> <http://wso2.com/about/team/malaka-silva/>
>>>> https://store.wso2.com/store/
>>>>
>>>> Don't make Trees rare, we should keep them with care
>>>>
>>>> _______________________________________________
>>>> Architecture mailing list
>>>> Architecture@wso2.org
>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
>>>>
>>>>
>>>
>>>
>>> --
>>> Malith Jayasinghe
>>>
>>> WSO2, Inc. (http://wso2.com)
>>> Email   : mali...@wso2.com
>>> Mobile : 0770704040
>>> Lean . Enterprise . Middleware
>>>
>>
>>
>>
>> --
>> Kathees
>> Software Engineer,
>> email: kath...@wso2.com
>> mobile: +94772596173 <+94%2077%20259%206173>
>>
>
>
>
> --
> Dushan Abeyruwan | Senior Technical Lead
> Technical Support,MV
> PMC Member Apache Synpase
> WSO2 Inc. http://wso2.com/
> Blog:*http://www.dushantech.com/ <http://www.dushantech.com/>*
> LinkedIn:*https://www.linkedin.com/in/dushanabeyruwan
> <https://www.linkedin.com/in/dushanabeyruwan>*
> Mobile:(001)408-791-9312
>
>


-- 
Kathees
Software Engineer,
email: kath...@wso2.com
mobile: +94772596173
_______________________________________________
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture

Reply via email to