On Tue, Sep 4, 2012 at 4:34 PM, Tharindu Mathew <[email protected]> wrote:
> The % seems to be quite acceptable after removing registry lookup. I will > commit this patch. I don't agree with this. 32% overhead for publishing stats is not acceptable. This indicates that the thread pool which is used for serving requests is being used for publishing stats, instead of getting the response back to the client as soon as possible. If you are doing such heavy lifting, it needs to be offloaded to some other thread pool, which will take care of publishing separately. I would consider this a major blocker for this release. > > > On Tue, Sep 4, 2012 at 3:56 PM, Sinthuja Ragendran <[email protected]>wrote: > >> Hi, >> >> I have changed the AS publisher to use the local caching. Since the >> branch is frozen i have attached the patch in JIRA [1]. >> >> I did the jprofiling after using the local caching, both profiles when >> enabling the publishing and not enabling the publishing are attached. >> >> [1] https://wso2.org/jira/browse/BAM-821 >> >> Thanks, >> Sinthuja. >> >> >> On Tue, Sep 4, 2012 at 1:58 PM, Afkham Azeez <[email protected]> wrote: >> >>> >>> >>> On Tue, Sep 4, 2012 at 1:33 PM, Srinath Perera <[email protected]> wrote: >>> >>>> Best is to check this at the time we create the handler and remember it >>> >>> >>> +1 >>> >>> >>> >>>> >>>> >>>> On Tue, Sep 4, 2012 at 1:11 PM, Tharindu Mathew <[email protected]>wrote: >>>> >>>>> The local caching should also solve the problem of the initial lookup >>>>> as well. That is we will only consult the registry to see whether stats or >>>>> activity is enabled ONCE, after that it has to just pass through. >>>>> >>>>> Note that the best approach is to not engage the handlers at all, >>>>> unless stats or activity monitoring is enabled. That should solve problems >>>>> when monitoring is not required, and the local caching would solve the >>>>> problem when monitoring is required. >>>>> >>>>> Sinthuja, after the fixes please profile and attach a diagram again. >>>>> >>>>> On Tue, Sep 4, 2012 at 12:54 PM, Sinthuja Ragendran <[email protected] >>>>> > wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> I profiled the as publishing. Since we are using the registry caching >>>>>> to save the configuration, it is taking more time to puublish. I have >>>>>> attached the screen shot of the profiling. >>>>>> >>>>>> As per offline discussion with azeez and tharindu, now we thought of >>>>>> going with local caching as earlier implementation. >>>>>> And now I'm working on this to reverting to back to previous >>>>>> implementation. >>>>>> >>>>>> Thanks, >>>>>> Sinthuja. >>>>>> >>>>>> >>>>>> On Tue, Sep 4, 2012 at 12:44 PM, Suhothayan Sriskandarajah < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> >>>>>>> >>>>>>> On Tue, Sep 4, 2012 at 12:00 PM, Srinath Perera <[email protected]>wrote: >>>>>>> >>>>>>>> IMHO, we need a urgent code review on this. Also get Suho. >>>>>>>> >>>>>>>> Sure, please let me know when the code review is. >>>>>>> >>>>>>> Earlier Appserver BAM publishers are not Async, hence Sinthuja wrote >>>>>>> a AsyncDataPruclihser to solve this issue. Now the data publishing >>>>>>> part is >>>>>>> very fast because all the requests are handled through a thread pool. >>>>>>> But as I know the data collection, and the event creation logic are >>>>>>> still not Async, this might me the reason, we have to look into this >>>>>>> more. >>>>>>> >>>>>>> Suho >>>>>>> >>>>>>> >>>>>>> We also saw ESB stat collectors looping. >>>>>>>> >>>>>>>> --Srinath >>>>>>>> >>>>>>>> On Tue, Sep 4, 2012 at 11:46 AM, Afkham Azeez <[email protected]>wrote: >>>>>>>> >>>>>>>>> We profiled AppServer 4.1.0 & compared it with the new AppServer >>>>>>>>> packs after removing the BAM publisher components. The old AppServer >>>>>>>>> 4.1.0 >>>>>>>>> is about 10-15% faster! >>>>>>>>> >>>>>>>>> >>>>>>>>> On Tue, Sep 4, 2012 at 11:13 AM, Afkham Azeez <[email protected]>wrote: >>>>>>>>> >>>>>>>>>> After removing the BAM publisher components, the throughput >>>>>>>>>> increased by 10 times! >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Tue, Sep 4, 2012 at 11:04 AM, Afkham Azeez <[email protected]>wrote: >>>>>>>>>> >>>>>>>>>>> Please take a look at the attached JProfiler screenshot. This is >>>>>>>>>>> the contributor to the 50% loss in throughout between the previous >>>>>>>>>>> AS >>>>>>>>>>> release and the current AS nightly build. >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> *Afkham Azeez* >>>>>>>>>>> Director of Architecture; WSO2, Inc.; http://wso2.com >>>>>>>>>>> Member; Apache Software Foundation; http://www.apache.org/ >>>>>>>>>>> * <http://www.apache.org/>** >>>>>>>>>>> email: **[email protected]* <[email protected]>* cell: +94 77 3320919 >>>>>>>>>>> blog: **http://blog.afkham.org* <http://blog.afkham.org>* >>>>>>>>>>> twitter: >>>>>>>>>>> **http://twitter.com/afkham_azeez*<http://twitter.com/afkham_azeez> >>>>>>>>>>> * >>>>>>>>>>> linked-in: **http://lk.linkedin.com/in/afkhamazeez* >>>>>>>>>>> * >>>>>>>>>>> * >>>>>>>>>>> *Lean . Enterprise . Middleware* >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> *Afkham Azeez* >>>>>>>>>> Director of Architecture; WSO2, Inc.; http://wso2.com >>>>>>>>>> Member; Apache Software Foundation; http://www.apache.org/ >>>>>>>>>> * <http://www.apache.org/>** >>>>>>>>>> email: **[email protected]* <[email protected]>* cell: +94 77 3320919 >>>>>>>>>> blog: **http://blog.afkham.org* <http://blog.afkham.org>* >>>>>>>>>> twitter: >>>>>>>>>> **http://twitter.com/afkham_azeez*<http://twitter.com/afkham_azeez> >>>>>>>>>> * >>>>>>>>>> linked-in: **http://lk.linkedin.com/in/afkhamazeez* >>>>>>>>>> * >>>>>>>>>> * >>>>>>>>>> *Lean . Enterprise . Middleware* >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> *Afkham Azeez* >>>>>>>>> Director of Architecture; WSO2, Inc.; http://wso2.com >>>>>>>>> Member; Apache Software Foundation; http://www.apache.org/ >>>>>>>>> * <http://www.apache.org/>** >>>>>>>>> email: **[email protected]* <[email protected]>* cell: +94 77 3320919 >>>>>>>>> blog: **http://blog.afkham.org* <http://blog.afkham.org>* >>>>>>>>> twitter: >>>>>>>>> **http://twitter.com/afkham_azeez*<http://twitter.com/afkham_azeez> >>>>>>>>> * >>>>>>>>> linked-in: **http://lk.linkedin.com/in/afkhamazeez* >>>>>>>>> * >>>>>>>>> * >>>>>>>>> *Lean . Enterprise . Middleware* >>>>>>>>> >>>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> Dev mailing list >>>>>>>>> [email protected] >>>>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> ============================ >>>>>>>> Srinath Perera, Ph.D. >>>>>>>> http://www.cs.indiana.edu/~hperera/ >>>>>>>> http://srinathsview.blogspot.com/ >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> Dev mailing list >>>>>>>> [email protected] >>>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> *S. Suhothayan >>>>>>> * >>>>>>> Software Engineer, >>>>>>> Data Technologies Team, >>>>>>> *WSO2, Inc. **http://wso2.com >>>>>>> <http://wso2.com/>* >>>>>>> *lean.enterprise.middleware.* >>>>>>> >>>>>>> *email: **[email protected]* <[email protected]>* cell: (+94) 779 756 757 >>>>>>> blog: >>>>>>> **http://suhothayan.blogspot.com/*<http://suhothayan.blogspot.com/> >>>>>>> * >>>>>>> twitter: **http://twitter.com/suhothayan*<http://twitter.com/suhothayan> >>>>>>> * >>>>>>> linked-in: **http://lk.linkedin.com/in/suhothayan* >>>>>>> * >>>>>>> * >>>>>>> >>>>>>> >>>>>>> _______________________________________________ >>>>>>> Dev mailing list >>>>>>> [email protected] >>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>>> >>>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> Regards, >>>>> >>>>> Tharindu >>>>> >>>>> blog: http://mackiemathew.com/ >>>>> M: +94777759908 >>>>> >>>>> >>>> >>>> >>>> -- >>>> ============================ >>>> Srinath Perera, Ph.D. >>>> Senior Software Architect, WSO2 Inc. >>>> Visiting Faculty, University of Moratuwa >>>> Member, Apache Software Foundation >>>> Research Scientist, Lanka Software Foundation >>>> Blog: http://srinathsview.blogspot.com/ >>>> Photos: http://www.flickr.com/photos/hemapani/ >>>> Phone: 0772360902 >>>> >>> >>> >>> >>> -- >>> *Afkham Azeez* >>> Director of Architecture; WSO2, Inc.; http://wso2.com >>> Member; Apache Software Foundation; http://www.apache.org/ >>> * <http://www.apache.org/>** >>> email: **[email protected]* <[email protected]>* cell: +94 77 3320919 >>> blog: **http://blog.afkham.org* <http://blog.afkham.org>* >>> twitter: **http://twitter.com/afkham_azeez*<http://twitter.com/afkham_azeez> >>> * >>> linked-in: **http://lk.linkedin.com/in/afkhamazeez* >>> * >>> * >>> *Lean . Enterprise . Middleware* >>> >>> >> > > > -- > Regards, > > Tharindu > > blog: http://mackiemathew.com/ > M: +94777759908 > > -- *Afkham Azeez* Director of Architecture; WSO2, Inc.; http://wso2.com Member; Apache Software Foundation; http://www.apache.org/ * <http://www.apache.org/>** email: **[email protected]* <[email protected]>* cell: +94 77 3320919 blog: **http://blog.afkham.org* <http://blog.afkham.org>* twitter: **http://twitter.com/afkham_azeez*<http://twitter.com/afkham_azeez> * linked-in: **http://lk.linkedin.com/in/afkhamazeez* * * *Lean . Enterprise . Middleware*
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
