Hi Kathees, Yes, there seems to be an obvious improvement. However, when comes to JMS connector,try to design that more generic, I guess that can be done with some customizations of what Hasitha has done.
Anyway, please do let me know when JMS genetic connector design planned, we can run through the design of that connector. Have you started to write JMS connector separately ? Cheers, Dushan On Wed, Nov 30, 2016 at 10:39 PM, Kathees Rajendram <kath...@wso2.com> wrote: > Hi Malaka, > > Yes. Once I compare this performance with other implementations and will > arrange a code review. > > @Malith > I will compare the performance of this custom connection pool > implementation with other implementations as it is mentioned in the mail > thread [1] that Stormpot throughput is highest, will check the > possibility to add in the connector and analyze performance results. > > [1] - "[Architecture] Benchmarks for Object Pool Implementations" > > Thanks, > Kathees > > On Thu, Dec 1, 2016 at 9:29 AM, Malith Jayasinghe <mali...@wso2.com> > wrote: > >> Hi Kathees, >> >> The underlying implementation of pool can have a significant impact on >> the performance. See the architecture tread "bench marks for object pool >> implementations" >> >> What is reason for going for a custom implementation here? Since this is >> a custom implementation it might be a good idea to do a micro-bench mark of >> the pool and compare its performance with other implementations. >> >> You can also profile the application to find out where the contentions >> etc are and then improve the performance (by narrowing down lock scopes >> etc) >> >> regards >> >> Malith >> >> On Wed, Nov 30, 2016 at 3:26 PM, Malaka Silva <mal...@wso2.com> wrote: >> >>> Seems like a significant improvement. >>> >>> I guess pool exists per node in cluster? >>> >>> Once completed please arrange a code review. >>> >>> On Wed, Nov 30, 2016 at 12:27 PM, Kathees Rajendram <kath...@wso2.com> >>> wrote: >>> >>>> Hi, >>>> >>>> I am working on connection pool implementation for Kafka [1] and JMS >>>> [2] connectors. In the existing connectors, connections are created >>>> and closed per message which gives the performance issue. I have improved >>>> the Kafka connector with configurable connection pool parameter. When >>>> publishing the messages, the maximum pool size parameter value can be >>>> changed in the connector configuration. Now the implementation with >>>> connection pool gives better performance. >>>> >>>> The following results are with connection pool implementation. >>>> Average throughput - 4980/s >>>> >>>> >>>> *Threads & Pool Size* *Messages per a Thread* *No of Messages* *A >>>> Message Size(byte)* >>>> *Throughput (/s)* >>>> 10 >>>> 500 5,000 106 3728 >>>> 20 500 10,000 106 4757 >>>> 100 >>>> 2500 250,000 106 5283 >>>> 100 10000 1,000,000 106 4920 >>>> >>>> >>>> The following results were without connection pool implementation. >>>> Average throughput - 2767/s >>>> >>>> *Threads* *Messages per a Thread* *No of Messages* *A Message >>>> Size(byte)* >>>> *Throughput (/s)* >>>> 10 500 5,000 106 1590 >>>> 20 500 10,000 106 2251 >>>> 100 250 25,000 106 2940 >>>> 1,000 1500 1,500,000 106 2780 >>>> In my git repo [3], I have committed the connection pool implementation >>>> for Kafka connector. Please give your suggestion. >>>> >>>> [1] - https://wso2.org/jira/browse/ESBCONNECT-122 >>>> [2] - https://wso2.org/jira/browse/ESBCONNECT-142 >>>> [3] - https://github.com/RKathees/esb-connector-kafka/blob/master/ >>>> src/main/java/org/wso2/carbon/connector/KafkaConnectionPoolManager.java >>>> >>>> Thanks, >>>> Kathees >>>> -- >>>> Kathees >>>> Software Engineer, >>>> email: kath...@wso2.com >>>> mobile: +94772596173 <077%20259%206173> >>>> >>> >>> >>> >>> -- >>> >>> Best Regards, >>> >>> Malaka Silva >>> Senior Technical Lead >>> M: +94 777 219 791 <+94%2077%20721%209791> >>> Tel : 94 11 214 5345 >>> Fax :94 11 2145300 >>> Skype : malaka.sampath.silva >>> LinkedIn : http://www.linkedin.com/pub/malaka-silva/6/33/77 >>> Blog : http://mrmalakasilva.blogspot.com/ >>> >>> WSO2, Inc. >>> lean . enterprise . middleware >>> https://wso2.com/signature >>> http://www.wso2.com/about/team/malaka-silva/ >>> <http://wso2.com/about/team/malaka-silva/> >>> https://store.wso2.com/store/ >>> >>> Don't make Trees rare, we should keep them with care >>> >>> _______________________________________________ >>> Architecture mailing list >>> Architecture@wso2.org >>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>> >>> >> >> >> -- >> Malith Jayasinghe >> >> WSO2, Inc. (http://wso2.com) >> Email : mali...@wso2.com >> Mobile : 0770704040 >> Lean . Enterprise . Middleware >> > > > > -- > Kathees > Software Engineer, > email: kath...@wso2.com > mobile: +94772596173 <+94%2077%20259%206173> > -- Dushan Abeyruwan | Senior Technical Lead Technical Support,MV PMC Member Apache Synpase WSO2 Inc. http://wso2.com/ Blog:*http://www.dushantech.com/ <http://www.dushantech.com/>* LinkedIn:*https://www.linkedin.com/in/dushanabeyruwan <https://www.linkedin.com/in/dushanabeyruwan>* Mobile:(001)408-791-9312
_______________________________________________ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
