Folks, We need to get this to under 1% overhead by tomorrow. Throughout today, we have made major performance improvements. Code segment have been improved to yield from 20% to less than 1% overhead on the request-response path. Several segments were tuned from 5-10% down to <1% overhead, so I believe the same can be done for BAM.
Azeez On Tue, Sep 4, 2012 at 9:47 PM, Afkham Azeez <az...@wso2.com> wrote: > Echo service. 40k requests. > > > On Tue, Sep 4, 2012 at 9:44 PM, Tharindu Mathew <thari...@wso2.com> wrote: > >> Sinthuja, >> >> Can you attach a profile with only stats enabled? The one you attached >> has both stats and activity enabled. There also seems to be some more >> improvements we can make to the data publisher side. We can offload more as >> Azeez suggests. >> >> Azeez, for how many requests did you profile for your result and to what >> service? >> >> >> >> On Tue, Sep 4, 2012 at 9:38 PM, Afkham Azeez <az...@wso2.com> wrote: >> >>> >>> >>> On Tue, Sep 4, 2012 at 4:34 PM, Tharindu Mathew <thari...@wso2.com>wrote: >>> >>>> The % seems to be quite acceptable after removing registry lookup. I >>>> will commit this patch. >>> >>> >>> I don't agree with this. 32% overhead for publishing stats is not >>> acceptable. This indicates that the thread pool which is used for serving >>> requests is being used for publishing stats, instead of getting the >>> response back to the client as soon as possible. If you are doing such >>> heavy lifting, it needs to be offloaded to some other thread pool, which >>> will take care of publishing separately. >>> >>> I would consider this a major blocker for this release. >>> >>> >>>> >>>> >>>> On Tue, Sep 4, 2012 at 3:56 PM, Sinthuja Ragendran >>>> <sinth...@wso2.com>wrote: >>>> >>>>> Hi, >>>>> >>>>> I have changed the AS publisher to use the local caching. Since the >>>>> branch is frozen i have attached the patch in JIRA [1]. >>>>> >>>>> I did the jprofiling after using the local caching, both profiles when >>>>> enabling the publishing and not enabling the publishing are attached. >>>>> >>>>> [1] https://wso2.org/jira/browse/BAM-821 >>>>> >>>>> Thanks, >>>>> Sinthuja. >>>>> >>>>> >>>>> On Tue, Sep 4, 2012 at 1:58 PM, Afkham Azeez <az...@wso2.com> wrote: >>>>> >>>>>> >>>>>> >>>>>> On Tue, Sep 4, 2012 at 1:33 PM, Srinath Perera <srin...@wso2.com>wrote: >>>>>> >>>>>>> Best is to check this at the time we create the handler and remember >>>>>>> it >>>>>> >>>>>> >>>>>> +1 >>>>>> >>>>>> >>>>>> >>>>>>> >>>>>>> >>>>>>> On Tue, Sep 4, 2012 at 1:11 PM, Tharindu Mathew >>>>>>> <thari...@wso2.com>wrote: >>>>>>> >>>>>>>> The local caching should also solve the problem of the initial >>>>>>>> lookup as well. That is we will only consult the registry to see >>>>>>>> whether >>>>>>>> stats or activity is enabled ONCE, after that it has to just pass >>>>>>>> through. >>>>>>>> >>>>>>>> Note that the best approach is to not engage the handlers at all, >>>>>>>> unless stats or activity monitoring is enabled. That should solve >>>>>>>> problems >>>>>>>> when monitoring is not required, and the local caching would solve the >>>>>>>> problem when monitoring is required. >>>>>>>> >>>>>>>> Sinthuja, after the fixes please profile and attach a diagram again. >>>>>>>> >>>>>>>> On Tue, Sep 4, 2012 at 12:54 PM, Sinthuja Ragendran < >>>>>>>> sinth...@wso2.com> wrote: >>>>>>>> >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> I profiled the as publishing. Since we are using the registry >>>>>>>>> caching to save the configuration, it is taking more time to >>>>>>>>> puublish. I >>>>>>>>> have attached the screen shot of the profiling. >>>>>>>>> >>>>>>>>> As per offline discussion with azeez and tharindu, now we thought >>>>>>>>> of going with local caching as earlier implementation. >>>>>>>>> And now I'm working on this to reverting to back to previous >>>>>>>>> implementation. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Sinthuja. >>>>>>>>> >>>>>>>>> >>>>>>>>> On Tue, Sep 4, 2012 at 12:44 PM, Suhothayan Sriskandarajah < >>>>>>>>> s...@wso2.com> wrote: >>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Tue, Sep 4, 2012 at 12:00 PM, Srinath Perera <srin...@wso2.com >>>>>>>>>> > wrote: >>>>>>>>>> >>>>>>>>>>> IMHO, we need a urgent code review on this. Also get Suho. >>>>>>>>>>> >>>>>>>>>>> Sure, please let me know when the code review is. >>>>>>>>>> >>>>>>>>>> Earlier Appserver BAM publishers are not Async, hence Sinthuja >>>>>>>>>> wrote a AsyncDataPruclihser to solve this issue. Now the data >>>>>>>>>> publishing >>>>>>>>>> part is very fast because all the requests are handled through a >>>>>>>>>> thread >>>>>>>>>> pool. >>>>>>>>>> But as I know the data collection, and the event creation logic >>>>>>>>>> are still not Async, this might me the reason, we have to look into >>>>>>>>>> this >>>>>>>>>> more. >>>>>>>>>> >>>>>>>>>> Suho >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> We also saw ESB stat collectors looping. >>>>>>>>>>> >>>>>>>>>>> --Srinath >>>>>>>>>>> >>>>>>>>>>> On Tue, Sep 4, 2012 at 11:46 AM, Afkham Azeez <az...@wso2.com>wrote: >>>>>>>>>>> >>>>>>>>>>>> We profiled AppServer 4.1.0 & compared it with the new >>>>>>>>>>>> AppServer packs after removing the BAM publisher components. The >>>>>>>>>>>> old >>>>>>>>>>>> AppServer 4.1.0 is about 10-15% faster! >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On Tue, Sep 4, 2012 at 11:13 AM, Afkham Azeez >>>>>>>>>>>> <az...@wso2.com>wrote: >>>>>>>>>>>> >>>>>>>>>>>>> After removing the BAM publisher components, the throughput >>>>>>>>>>>>> increased by 10 times! >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On Tue, Sep 4, 2012 at 11:04 AM, Afkham Azeez >>>>>>>>>>>>> <az...@wso2.com>wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Please take a look at the attached JProfiler screenshot. This >>>>>>>>>>>>>> is the contributor to the 50% loss in throughout between the >>>>>>>>>>>>>> previous AS >>>>>>>>>>>>>> release and the current AS nightly build. >>>>>>>>>>>>>> >>>>>>>>>>>>>> -- >>>>>>>>>>>>>> *Afkham Azeez* >>>>>>>>>>>>>> Director of Architecture; WSO2, Inc.; http://wso2.com >>>>>>>>>>>>>> Member; Apache Software Foundation; http://www.apache.org/ >>>>>>>>>>>>>> * <http://www.apache.org/>** >>>>>>>>>>>>>> email: **az...@wso2.com* <az...@wso2.com>* cell: +94 77 >>>>>>>>>>>>>> 3320919 >>>>>>>>>>>>>> blog: **http://blog.afkham.org* <http://blog.afkham.org>* >>>>>>>>>>>>>> twitter: >>>>>>>>>>>>>> **http://twitter.com/afkham_azeez*<http://twitter.com/afkham_azeez> >>>>>>>>>>>>>> * >>>>>>>>>>>>>> linked-in: **http://lk.linkedin.com/in/afkhamazeez* >>>>>>>>>>>>>> * >>>>>>>>>>>>>> * >>>>>>>>>>>>>> *Lean . Enterprise . Middleware* >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> -- >>>>>>>>>>>>> *Afkham Azeez* >>>>>>>>>>>>> Director of Architecture; WSO2, Inc.; http://wso2.com >>>>>>>>>>>>> Member; Apache Software Foundation; http://www.apache.org/ >>>>>>>>>>>>> * <http://www.apache.org/>** >>>>>>>>>>>>> email: **az...@wso2.com* <az...@wso2.com>* cell: +94 77 >>>>>>>>>>>>> 3320919 >>>>>>>>>>>>> blog: **http://blog.afkham.org* <http://blog.afkham.org>* >>>>>>>>>>>>> twitter: >>>>>>>>>>>>> **http://twitter.com/afkham_azeez*<http://twitter.com/afkham_azeez> >>>>>>>>>>>>> * >>>>>>>>>>>>> linked-in: **http://lk.linkedin.com/in/afkhamazeez* >>>>>>>>>>>>> * >>>>>>>>>>>>> * >>>>>>>>>>>>> *Lean . Enterprise . Middleware* >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> *Afkham Azeez* >>>>>>>>>>>> Director of Architecture; WSO2, Inc.; http://wso2.com >>>>>>>>>>>> Member; Apache Software Foundation; http://www.apache.org/ >>>>>>>>>>>> * <http://www.apache.org/>** >>>>>>>>>>>> email: **az...@wso2.com* <az...@wso2.com>* cell: +94 77 3320919 >>>>>>>>>>>> blog: **http://blog.afkham.org* <http://blog.afkham.org>* >>>>>>>>>>>> twitter: >>>>>>>>>>>> **http://twitter.com/afkham_azeez*<http://twitter.com/afkham_azeez> >>>>>>>>>>>> * >>>>>>>>>>>> linked-in: **http://lk.linkedin.com/in/afkhamazeez* >>>>>>>>>>>> * >>>>>>>>>>>> * >>>>>>>>>>>> *Lean . Enterprise . Middleware* >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>> Dev mailing list >>>>>>>>>>>> Dev@wso2.org >>>>>>>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> ============================ >>>>>>>>>>> Srinath Perera, Ph.D. >>>>>>>>>>> http://www.cs.indiana.edu/~hperera/ >>>>>>>>>>> http://srinathsview.blogspot.com/ >>>>>>>>>>> >>>>>>>>>>> _______________________________________________ >>>>>>>>>>> Dev mailing list >>>>>>>>>>> Dev@wso2.org >>>>>>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> *S. Suhothayan >>>>>>>>>> * >>>>>>>>>> Software Engineer, >>>>>>>>>> Data Technologies Team, >>>>>>>>>> *WSO2, Inc. **http://wso2.com >>>>>>>>>> <http://wso2.com/>* >>>>>>>>>> *lean.enterprise.middleware.* >>>>>>>>>> >>>>>>>>>> *email: **s...@wso2.com* <s...@wso2.com>* cell: (+94) 779 756 757 >>>>>>>>>> blog: >>>>>>>>>> **http://suhothayan.blogspot.com/*<http://suhothayan.blogspot.com/> >>>>>>>>>> * >>>>>>>>>> twitter: >>>>>>>>>> **http://twitter.com/suhothayan*<http://twitter.com/suhothayan> >>>>>>>>>> * >>>>>>>>>> linked-in: **http://lk.linkedin.com/in/suhothayan* >>>>>>>>>> * >>>>>>>>>> * >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> _______________________________________________ >>>>>>>>>> Dev mailing list >>>>>>>>>> Dev@wso2.org >>>>>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Regards, >>>>>>>> >>>>>>>> Tharindu >>>>>>>> >>>>>>>> blog: http://mackiemathew.com/ >>>>>>>> M: +94777759908 >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> ============================ >>>>>>> Srinath Perera, Ph.D. >>>>>>> Senior Software Architect, WSO2 Inc. >>>>>>> Visiting Faculty, University of Moratuwa >>>>>>> Member, Apache Software Foundation >>>>>>> Research Scientist, Lanka Software Foundation >>>>>>> Blog: http://srinathsview.blogspot.com/ >>>>>>> Photos: http://www.flickr.com/photos/hemapani/ >>>>>>> Phone: 0772360902 >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> *Afkham Azeez* >>>>>> Director of Architecture; WSO2, Inc.; http://wso2.com >>>>>> Member; Apache Software Foundation; http://www.apache.org/ >>>>>> * <http://www.apache.org/>** >>>>>> email: **az...@wso2.com* <az...@wso2.com>* cell: +94 77 3320919 >>>>>> blog: **http://blog.afkham.org* <http://blog.afkham.org>* >>>>>> twitter: >>>>>> **http://twitter.com/afkham_azeez*<http://twitter.com/afkham_azeez> >>>>>> * >>>>>> linked-in: **http://lk.linkedin.com/in/afkhamazeez* >>>>>> * >>>>>> * >>>>>> *Lean . Enterprise . Middleware* >>>>>> >>>>>> >>>>> >>>> >>>> >>>> -- >>>> Regards, >>>> >>>> Tharindu >>>> >>>> blog: http://mackiemathew.com/ >>>> M: +94777759908 >>>> >>>> >>> >>> >>> -- >>> *Afkham Azeez* >>> Director of Architecture; WSO2, Inc.; http://wso2.com >>> Member; Apache Software Foundation; http://www.apache.org/ >>> * <http://www.apache.org/>** >>> email: **az...@wso2.com* <az...@wso2.com>* cell: +94 77 3320919 >>> blog: **http://blog.afkham.org* <http://blog.afkham.org>* >>> twitter: **http://twitter.com/afkham_azeez*<http://twitter.com/afkham_azeez> >>> * >>> linked-in: **http://lk.linkedin.com/in/afkhamazeez* >>> * >>> * >>> *Lean . Enterprise . Middleware* >>> >>> >> >> >> -- >> Regards, >> >> Tharindu >> >> blog: http://mackiemathew.com/ >> M: +94777759908 >> >> > > > -- > *Afkham Azeez* > Director of Architecture; WSO2, Inc.; http://wso2.com > Member; Apache Software Foundation; http://www.apache.org/ > * <http://www.apache.org/>** > email: **az...@wso2.com* <az...@wso2.com>* cell: +94 77 3320919 > blog: **http://blog.afkham.org* <http://blog.afkham.org>* > twitter: **http://twitter.com/afkham_azeez*<http://twitter.com/afkham_azeez> > * > linked-in: **http://lk.linkedin.com/in/afkhamazeez* > * > * > *Lean . Enterprise . Middleware* > > -- *Afkham Azeez* Director of Architecture; WSO2, Inc.; http://wso2.com Member; Apache Software Foundation; http://www.apache.org/ * <http://www.apache.org/>** email: **az...@wso2.com* <az...@wso2.com>* cell: +94 77 3320919 blog: **http://blog.afkham.org* <http://blog.afkham.org>* twitter: **http://twitter.com/afkham_azeez*<http://twitter.com/afkham_azeez> * linked-in: **http://lk.linkedin.com/in/afkhamazeez* * * *Lean . Enterprise . Middleware*
_______________________________________________ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
