Hi Dushan, I am working on connection pool implementation for Kafka Connector. By next week, I am going to start implementing the JMS connector. I will let you know while designing the connector.
Thanks, Kathees On Thu, Dec 1, 2016 at 12:34 PM, Dushan Abeyruwan <[email protected]> wrote: > Hi Kathees, > Yes, there seems to be an obvious improvement. However, when comes to > JMS connector,try to design that more generic, I guess that can be done > with some customizations of what Hasitha has done. > > Anyway, please do let me know when JMS genetic connector design planned, > we can run through the design of that connector. > > Have you started to write JMS connector separately ? > > Cheers, > Dushan > > On Wed, Nov 30, 2016 at 10:39 PM, Kathees Rajendram <[email protected]> > wrote: > >> Hi Malaka, >> >> Yes. Once I compare this performance with other implementations and will >> arrange a code review. >> >> @Malith >> I will compare the performance of this custom connection pool >> implementation with other implementations as it is mentioned in the mail >> thread [1] that Stormpot throughput is highest, will check the >> possibility to add in the connector and analyze performance results. >> >> [1] - "[Architecture] Benchmarks for Object Pool Implementations" >> >> Thanks, >> Kathees >> >> On Thu, Dec 1, 2016 at 9:29 AM, Malith Jayasinghe <[email protected]> >> wrote: >> >>> Hi Kathees, >>> >>> The underlying implementation of pool can have a significant impact on >>> the performance. See the architecture tread "bench marks for object pool >>> implementations" >>> >>> What is reason for going for a custom implementation here? Since this is >>> a custom implementation it might be a good idea to do a micro-bench mark of >>> the pool and compare its performance with other implementations. >>> >>> You can also profile the application to find out where the contentions >>> etc are and then improve the performance (by narrowing down lock scopes >>> etc) >>> >>> regards >>> >>> Malith >>> >>> On Wed, Nov 30, 2016 at 3:26 PM, Malaka Silva <[email protected]> wrote: >>> >>>> Seems like a significant improvement. >>>> >>>> I guess pool exists per node in cluster? >>>> >>>> Once completed please arrange a code review. >>>> >>>> On Wed, Nov 30, 2016 at 12:27 PM, Kathees Rajendram <[email protected]> >>>> wrote: >>>> >>>>> Hi, >>>>> >>>>> I am working on connection pool implementation for Kafka [1] and JMS >>>>> [2] connectors. In the existing connectors, connections are created >>>>> and closed per message which gives the performance issue. I have improved >>>>> the Kafka connector with configurable connection pool parameter. When >>>>> publishing the messages, the maximum pool size parameter value can be >>>>> changed in the connector configuration. Now the implementation with >>>>> connection pool gives better performance. >>>>> >>>>> The following results are with connection pool implementation. >>>>> Average throughput - 4980/s >>>>> >>>>> >>>>> *Threads & Pool Size* *Messages per a Thread* *No of Messages* *A >>>>> Message Size(byte)* >>>>> *Throughput (/s)* >>>>> 10 >>>>> 500 5,000 106 3728 >>>>> 20 500 10,000 106 4757 >>>>> 100 >>>>> 2500 250,000 106 5283 >>>>> 100 10000 1,000,000 106 4920 >>>>> >>>>> >>>>> The following results were without connection pool implementation. >>>>> Average throughput - 2767/s >>>>> >>>>> *Threads* *Messages per a Thread* *No of Messages* *A Message >>>>> Size(byte)* >>>>> *Throughput (/s)* >>>>> 10 500 5,000 106 1590 >>>>> 20 500 10,000 106 2251 >>>>> 100 250 25,000 106 2940 >>>>> 1,000 1500 1,500,000 106 2780 >>>>> In my git repo [3], I have committed the connection pool >>>>> implementation for Kafka connector. Please give your suggestion. >>>>> >>>>> [1] - https://wso2.org/jira/browse/ESBCONNECT-122 >>>>> [2] - https://wso2.org/jira/browse/ESBCONNECT-142 >>>>> [3] - https://github.com/RKathees/esb-connector-kafka/blob/master/ >>>>> src/main/java/org/wso2/carbon/connector/KafkaConnectionPoolM >>>>> anager.java >>>>> >>>>> Thanks, >>>>> Kathees >>>>> -- >>>>> Kathees >>>>> Software Engineer, >>>>> email: [email protected] >>>>> mobile: +94772596173 <077%20259%206173> >>>>> >>>> >>>> >>>> >>>> -- >>>> >>>> Best Regards, >>>> >>>> Malaka Silva >>>> Senior Technical Lead >>>> M: +94 777 219 791 <+94%2077%20721%209791> >>>> Tel : 94 11 214 5345 >>>> Fax :94 11 2145300 >>>> Skype : malaka.sampath.silva >>>> LinkedIn : http://www.linkedin.com/pub/malaka-silva/6/33/77 >>>> Blog : http://mrmalakasilva.blogspot.com/ >>>> >>>> WSO2, Inc. >>>> lean . enterprise . middleware >>>> https://wso2.com/signature >>>> http://www.wso2.com/about/team/malaka-silva/ >>>> <http://wso2.com/about/team/malaka-silva/> >>>> https://store.wso2.com/store/ >>>> >>>> Don't make Trees rare, we should keep them with care >>>> >>>> _______________________________________________ >>>> Architecture mailing list >>>> [email protected] >>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>> >>>> >>> >>> >>> -- >>> Malith Jayasinghe >>> >>> WSO2, Inc. (http://wso2.com) >>> Email : [email protected] >>> Mobile : 0770704040 >>> Lean . Enterprise . Middleware >>> >> >> >> >> -- >> Kathees >> Software Engineer, >> email: [email protected] >> mobile: +94772596173 <+94%2077%20259%206173> >> > > > > -- > Dushan Abeyruwan | Senior Technical Lead > Technical Support,MV > PMC Member Apache Synpase > WSO2 Inc. http://wso2.com/ > Blog:*http://www.dushantech.com/ <http://www.dushantech.com/>* > LinkedIn:*https://www.linkedin.com/in/dushanabeyruwan > <https://www.linkedin.com/in/dushanabeyruwan>* > Mobile:(001)408-791-9312 > > -- Kathees Software Engineer, email: [email protected] mobile: +94772596173
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
