Sinthuja, Can you attach a profile with only stats enabled? The one you attached has both stats and activity enabled. There also seems to be some more improvements we can make to the data publisher side. We can offload more as Azeez suggests.
Azeez, for how many requests did you profile for your result and to what service? On Tue, Sep 4, 2012 at 9:38 PM, Afkham Azeez <az...@wso2.com> wrote: > > > On Tue, Sep 4, 2012 at 4:34 PM, Tharindu Mathew <thari...@wso2.com> wrote: > >> The % seems to be quite acceptable after removing registry lookup. I will >> commit this patch. > > > I don't agree with this. 32% overhead for publishing stats is not > acceptable. This indicates that the thread pool which is used for serving > requests is being used for publishing stats, instead of getting the > response back to the client as soon as possible. If you are doing such > heavy lifting, it needs to be offloaded to some other thread pool, which > will take care of publishing separately. > > I would consider this a major blocker for this release. > > >> >> >> On Tue, Sep 4, 2012 at 3:56 PM, Sinthuja Ragendran <sinth...@wso2.com>wrote: >> >>> Hi, >>> >>> I have changed the AS publisher to use the local caching. Since the >>> branch is frozen i have attached the patch in JIRA [1]. >>> >>> I did the jprofiling after using the local caching, both profiles when >>> enabling the publishing and not enabling the publishing are attached. >>> >>> [1] https://wso2.org/jira/browse/BAM-821 >>> >>> Thanks, >>> Sinthuja. >>> >>> >>> On Tue, Sep 4, 2012 at 1:58 PM, Afkham Azeez <az...@wso2.com> wrote: >>> >>>> >>>> >>>> On Tue, Sep 4, 2012 at 1:33 PM, Srinath Perera <srin...@wso2.com>wrote: >>>> >>>>> Best is to check this at the time we create the handler and remember >>>>> it >>>> >>>> >>>> +1 >>>> >>>> >>>> >>>>> >>>>> >>>>> On Tue, Sep 4, 2012 at 1:11 PM, Tharindu Mathew <thari...@wso2.com>wrote: >>>>> >>>>>> The local caching should also solve the problem of the initial lookup >>>>>> as well. That is we will only consult the registry to see whether stats >>>>>> or >>>>>> activity is enabled ONCE, after that it has to just pass through. >>>>>> >>>>>> Note that the best approach is to not engage the handlers at all, >>>>>> unless stats or activity monitoring is enabled. That should solve >>>>>> problems >>>>>> when monitoring is not required, and the local caching would solve the >>>>>> problem when monitoring is required. >>>>>> >>>>>> Sinthuja, after the fixes please profile and attach a diagram again. >>>>>> >>>>>> On Tue, Sep 4, 2012 at 12:54 PM, Sinthuja Ragendran < >>>>>> sinth...@wso2.com> wrote: >>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> I profiled the as publishing. Since we are using the registry >>>>>>> caching to save the configuration, it is taking more time to puublish. I >>>>>>> have attached the screen shot of the profiling. >>>>>>> >>>>>>> As per offline discussion with azeez and tharindu, now we thought of >>>>>>> going with local caching as earlier implementation. >>>>>>> And now I'm working on this to reverting to back to previous >>>>>>> implementation. >>>>>>> >>>>>>> Thanks, >>>>>>> Sinthuja. >>>>>>> >>>>>>> >>>>>>> On Tue, Sep 4, 2012 at 12:44 PM, Suhothayan Sriskandarajah < >>>>>>> s...@wso2.com> wrote: >>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Tue, Sep 4, 2012 at 12:00 PM, Srinath Perera >>>>>>>> <srin...@wso2.com>wrote: >>>>>>>> >>>>>>>>> IMHO, we need a urgent code review on this. Also get Suho. >>>>>>>>> >>>>>>>>> Sure, please let me know when the code review is. >>>>>>>> >>>>>>>> Earlier Appserver BAM publishers are not Async, hence Sinthuja >>>>>>>> wrote a AsyncDataPruclihser to solve this issue. Now the data >>>>>>>> publishing >>>>>>>> part is very fast because all the requests are handled through a thread >>>>>>>> pool. >>>>>>>> But as I know the data collection, and the event creation logic are >>>>>>>> still not Async, this might me the reason, we have to look into this >>>>>>>> more. >>>>>>>> >>>>>>>> Suho >>>>>>>> >>>>>>>> >>>>>>>> We also saw ESB stat collectors looping. >>>>>>>>> >>>>>>>>> --Srinath >>>>>>>>> >>>>>>>>> On Tue, Sep 4, 2012 at 11:46 AM, Afkham Azeez <az...@wso2.com>wrote: >>>>>>>>> >>>>>>>>>> We profiled AppServer 4.1.0 & compared it with the new AppServer >>>>>>>>>> packs after removing the BAM publisher components. The old AppServer >>>>>>>>>> 4.1.0 >>>>>>>>>> is about 10-15% faster! >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Tue, Sep 4, 2012 at 11:13 AM, Afkham Azeez <az...@wso2.com>wrote: >>>>>>>>>> >>>>>>>>>>> After removing the BAM publisher components, the throughput >>>>>>>>>>> increased by 10 times! >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Tue, Sep 4, 2012 at 11:04 AM, Afkham Azeez <az...@wso2.com>wrote: >>>>>>>>>>> >>>>>>>>>>>> Please take a look at the attached JProfiler screenshot. This >>>>>>>>>>>> is the contributor to the 50% loss in throughout between the >>>>>>>>>>>> previous AS >>>>>>>>>>>> release and the current AS nightly build. >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> *Afkham Azeez* >>>>>>>>>>>> Director of Architecture; WSO2, Inc.; http://wso2.com >>>>>>>>>>>> Member; Apache Software Foundation; http://www.apache.org/ >>>>>>>>>>>> * <http://www.apache.org/>** >>>>>>>>>>>> email: **az...@wso2.com* <az...@wso2.com>* cell: +94 77 3320919 >>>>>>>>>>>> blog: **http://blog.afkham.org* <http://blog.afkham.org>* >>>>>>>>>>>> twitter: >>>>>>>>>>>> **http://twitter.com/afkham_azeez*<http://twitter.com/afkham_azeez> >>>>>>>>>>>> * >>>>>>>>>>>> linked-in: **http://lk.linkedin.com/in/afkhamazeez* >>>>>>>>>>>> * >>>>>>>>>>>> * >>>>>>>>>>>> *Lean . Enterprise . Middleware* >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> *Afkham Azeez* >>>>>>>>>>> Director of Architecture; WSO2, Inc.; http://wso2.com >>>>>>>>>>> Member; Apache Software Foundation; http://www.apache.org/ >>>>>>>>>>> * <http://www.apache.org/>** >>>>>>>>>>> email: **az...@wso2.com* <az...@wso2.com>* cell: +94 77 3320919 >>>>>>>>>>> blog: **http://blog.afkham.org* <http://blog.afkham.org>* >>>>>>>>>>> twitter: >>>>>>>>>>> **http://twitter.com/afkham_azeez*<http://twitter.com/afkham_azeez> >>>>>>>>>>> * >>>>>>>>>>> linked-in: **http://lk.linkedin.com/in/afkhamazeez* >>>>>>>>>>> * >>>>>>>>>>> * >>>>>>>>>>> *Lean . Enterprise . Middleware* >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> *Afkham Azeez* >>>>>>>>>> Director of Architecture; WSO2, Inc.; http://wso2.com >>>>>>>>>> Member; Apache Software Foundation; http://www.apache.org/ >>>>>>>>>> * <http://www.apache.org/>** >>>>>>>>>> email: **az...@wso2.com* <az...@wso2.com>* cell: +94 77 3320919 >>>>>>>>>> blog: **http://blog.afkham.org* <http://blog.afkham.org>* >>>>>>>>>> twitter: >>>>>>>>>> **http://twitter.com/afkham_azeez*<http://twitter.com/afkham_azeez> >>>>>>>>>> * >>>>>>>>>> linked-in: **http://lk.linkedin.com/in/afkhamazeez* >>>>>>>>>> * >>>>>>>>>> * >>>>>>>>>> *Lean . Enterprise . Middleware* >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> _______________________________________________ >>>>>>>>>> Dev mailing list >>>>>>>>>> Dev@wso2.org >>>>>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> ============================ >>>>>>>>> Srinath Perera, Ph.D. >>>>>>>>> http://www.cs.indiana.edu/~hperera/ >>>>>>>>> http://srinathsview.blogspot.com/ >>>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> Dev mailing list >>>>>>>>> Dev@wso2.org >>>>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> *S. Suhothayan >>>>>>>> * >>>>>>>> Software Engineer, >>>>>>>> Data Technologies Team, >>>>>>>> *WSO2, Inc. **http://wso2.com >>>>>>>> <http://wso2.com/>* >>>>>>>> *lean.enterprise.middleware.* >>>>>>>> >>>>>>>> *email: **s...@wso2.com* <s...@wso2.com>* cell: (+94) 779 756 757 >>>>>>>> blog: >>>>>>>> **http://suhothayan.blogspot.com/*<http://suhothayan.blogspot.com/> >>>>>>>> * >>>>>>>> twitter: >>>>>>>> **http://twitter.com/suhothayan*<http://twitter.com/suhothayan> >>>>>>>> * >>>>>>>> linked-in: **http://lk.linkedin.com/in/suhothayan* >>>>>>>> * >>>>>>>> * >>>>>>>> >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> Dev mailing list >>>>>>>> Dev@wso2.org >>>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Regards, >>>>>> >>>>>> Tharindu >>>>>> >>>>>> blog: http://mackiemathew.com/ >>>>>> M: +94777759908 >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> ============================ >>>>> Srinath Perera, Ph.D. >>>>> Senior Software Architect, WSO2 Inc. >>>>> Visiting Faculty, University of Moratuwa >>>>> Member, Apache Software Foundation >>>>> Research Scientist, Lanka Software Foundation >>>>> Blog: http://srinathsview.blogspot.com/ >>>>> Photos: http://www.flickr.com/photos/hemapani/ >>>>> Phone: 0772360902 >>>>> >>>> >>>> >>>> >>>> -- >>>> *Afkham Azeez* >>>> Director of Architecture; WSO2, Inc.; http://wso2.com >>>> Member; Apache Software Foundation; http://www.apache.org/ >>>> * <http://www.apache.org/>** >>>> email: **az...@wso2.com* <az...@wso2.com>* cell: +94 77 3320919 >>>> blog: **http://blog.afkham.org* <http://blog.afkham.org>* >>>> twitter: >>>> **http://twitter.com/afkham_azeez*<http://twitter.com/afkham_azeez> >>>> * >>>> linked-in: **http://lk.linkedin.com/in/afkhamazeez* >>>> * >>>> * >>>> *Lean . Enterprise . Middleware* >>>> >>>> >>> >> >> >> -- >> Regards, >> >> Tharindu >> >> blog: http://mackiemathew.com/ >> M: +94777759908 >> >> > > > -- > *Afkham Azeez* > Director of Architecture; WSO2, Inc.; http://wso2.com > Member; Apache Software Foundation; http://www.apache.org/ > * <http://www.apache.org/>** > email: **az...@wso2.com* <az...@wso2.com>* cell: +94 77 3320919 > blog: **http://blog.afkham.org* <http://blog.afkham.org>* > twitter: **http://twitter.com/afkham_azeez*<http://twitter.com/afkham_azeez> > * > linked-in: **http://lk.linkedin.com/in/afkhamazeez* > * > * > *Lean . Enterprise . Middleware* > > -- Regards, Tharindu blog: http://mackiemathew.com/ M: +94777759908
_______________________________________________ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
