Echo service. 40k requests. On Tue, Sep 4, 2012 at 9:44 PM, Tharindu Mathew <[email protected]> wrote:
> Sinthuja, > > Can you attach a profile with only stats enabled? The one you attached has > both stats and activity enabled. There also seems to be some more > improvements we can make to the data publisher side. We can offload more as > Azeez suggests. > > Azeez, for how many requests did you profile for your result and to what > service? > > > > On Tue, Sep 4, 2012 at 9:38 PM, Afkham Azeez <[email protected]> wrote: > >> >> >> On Tue, Sep 4, 2012 at 4:34 PM, Tharindu Mathew <[email protected]>wrote: >> >>> The % seems to be quite acceptable after removing registry lookup. I >>> will commit this patch. >> >> >> I don't agree with this. 32% overhead for publishing stats is not >> acceptable. This indicates that the thread pool which is used for serving >> requests is being used for publishing stats, instead of getting the >> response back to the client as soon as possible. If you are doing such >> heavy lifting, it needs to be offloaded to some other thread pool, which >> will take care of publishing separately. >> >> I would consider this a major blocker for this release. >> >> >>> >>> >>> On Tue, Sep 4, 2012 at 3:56 PM, Sinthuja Ragendran <[email protected]>wrote: >>> >>>> Hi, >>>> >>>> I have changed the AS publisher to use the local caching. Since the >>>> branch is frozen i have attached the patch in JIRA [1]. >>>> >>>> I did the jprofiling after using the local caching, both profiles when >>>> enabling the publishing and not enabling the publishing are attached. >>>> >>>> [1] https://wso2.org/jira/browse/BAM-821 >>>> >>>> Thanks, >>>> Sinthuja. >>>> >>>> >>>> On Tue, Sep 4, 2012 at 1:58 PM, Afkham Azeez <[email protected]> wrote: >>>> >>>>> >>>>> >>>>> On Tue, Sep 4, 2012 at 1:33 PM, Srinath Perera <[email protected]>wrote: >>>>> >>>>>> Best is to check this at the time we create the handler and remember >>>>>> it >>>>> >>>>> >>>>> +1 >>>>> >>>>> >>>>> >>>>>> >>>>>> >>>>>> On Tue, Sep 4, 2012 at 1:11 PM, Tharindu Mathew <[email protected]>wrote: >>>>>> >>>>>>> The local caching should also solve the problem of the initial >>>>>>> lookup as well. That is we will only consult the registry to see whether >>>>>>> stats or activity is enabled ONCE, after that it has to just pass >>>>>>> through. >>>>>>> >>>>>>> Note that the best approach is to not engage the handlers at all, >>>>>>> unless stats or activity monitoring is enabled. That should solve >>>>>>> problems >>>>>>> when monitoring is not required, and the local caching would solve the >>>>>>> problem when monitoring is required. >>>>>>> >>>>>>> Sinthuja, after the fixes please profile and attach a diagram again. >>>>>>> >>>>>>> On Tue, Sep 4, 2012 at 12:54 PM, Sinthuja Ragendran < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> I profiled the as publishing. Since we are using the registry >>>>>>>> caching to save the configuration, it is taking more time to puublish. >>>>>>>> I >>>>>>>> have attached the screen shot of the profiling. >>>>>>>> >>>>>>>> As per offline discussion with azeez and tharindu, now we thought >>>>>>>> of going with local caching as earlier implementation. >>>>>>>> And now I'm working on this to reverting to back to previous >>>>>>>> implementation. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Sinthuja. >>>>>>>> >>>>>>>> >>>>>>>> On Tue, Sep 4, 2012 at 12:44 PM, Suhothayan Sriskandarajah < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On Tue, Sep 4, 2012 at 12:00 PM, Srinath Perera >>>>>>>>> <[email protected]>wrote: >>>>>>>>> >>>>>>>>>> IMHO, we need a urgent code review on this. Also get Suho. >>>>>>>>>> >>>>>>>>>> Sure, please let me know when the code review is. >>>>>>>>> >>>>>>>>> Earlier Appserver BAM publishers are not Async, hence Sinthuja >>>>>>>>> wrote a AsyncDataPruclihser to solve this issue. Now the data >>>>>>>>> publishing >>>>>>>>> part is very fast because all the requests are handled through a >>>>>>>>> thread >>>>>>>>> pool. >>>>>>>>> But as I know the data collection, and the event creation logic >>>>>>>>> are still not Async, this might me the reason, we have to look into >>>>>>>>> this >>>>>>>>> more. >>>>>>>>> >>>>>>>>> Suho >>>>>>>>> >>>>>>>>> >>>>>>>>> We also saw ESB stat collectors looping. >>>>>>>>>> >>>>>>>>>> --Srinath >>>>>>>>>> >>>>>>>>>> On Tue, Sep 4, 2012 at 11:46 AM, Afkham Azeez <[email protected]>wrote: >>>>>>>>>> >>>>>>>>>>> We profiled AppServer 4.1.0 & compared it with the new AppServer >>>>>>>>>>> packs after removing the BAM publisher components. The old >>>>>>>>>>> AppServer 4.1.0 >>>>>>>>>>> is about 10-15% faster! >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Tue, Sep 4, 2012 at 11:13 AM, Afkham Azeez <[email protected]>wrote: >>>>>>>>>>> >>>>>>>>>>>> After removing the BAM publisher components, the throughput >>>>>>>>>>>> increased by 10 times! >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On Tue, Sep 4, 2012 at 11:04 AM, Afkham Azeez >>>>>>>>>>>> <[email protected]>wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Please take a look at the attached JProfiler screenshot. This >>>>>>>>>>>>> is the contributor to the 50% loss in throughout between the >>>>>>>>>>>>> previous AS >>>>>>>>>>>>> release and the current AS nightly build. >>>>>>>>>>>>> >>>>>>>>>>>>> -- >>>>>>>>>>>>> *Afkham Azeez* >>>>>>>>>>>>> Director of Architecture; WSO2, Inc.; http://wso2.com >>>>>>>>>>>>> Member; Apache Software Foundation; http://www.apache.org/ >>>>>>>>>>>>> * <http://www.apache.org/>** >>>>>>>>>>>>> email: **[email protected]* <[email protected]>* cell: +94 77 >>>>>>>>>>>>> 3320919 >>>>>>>>>>>>> blog: **http://blog.afkham.org* <http://blog.afkham.org>* >>>>>>>>>>>>> twitter: >>>>>>>>>>>>> **http://twitter.com/afkham_azeez*<http://twitter.com/afkham_azeez> >>>>>>>>>>>>> * >>>>>>>>>>>>> linked-in: **http://lk.linkedin.com/in/afkhamazeez* >>>>>>>>>>>>> * >>>>>>>>>>>>> * >>>>>>>>>>>>> *Lean . Enterprise . Middleware* >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> *Afkham Azeez* >>>>>>>>>>>> Director of Architecture; WSO2, Inc.; http://wso2.com >>>>>>>>>>>> Member; Apache Software Foundation; http://www.apache.org/ >>>>>>>>>>>> * <http://www.apache.org/>** >>>>>>>>>>>> email: **[email protected]* <[email protected]>* cell: +94 77 3320919 >>>>>>>>>>>> blog: **http://blog.afkham.org* <http://blog.afkham.org>* >>>>>>>>>>>> twitter: >>>>>>>>>>>> **http://twitter.com/afkham_azeez*<http://twitter.com/afkham_azeez> >>>>>>>>>>>> * >>>>>>>>>>>> linked-in: **http://lk.linkedin.com/in/afkhamazeez* >>>>>>>>>>>> * >>>>>>>>>>>> * >>>>>>>>>>>> *Lean . Enterprise . Middleware* >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> *Afkham Azeez* >>>>>>>>>>> Director of Architecture; WSO2, Inc.; http://wso2.com >>>>>>>>>>> Member; Apache Software Foundation; http://www.apache.org/ >>>>>>>>>>> * <http://www.apache.org/>** >>>>>>>>>>> email: **[email protected]* <[email protected]>* cell: +94 77 3320919 >>>>>>>>>>> blog: **http://blog.afkham.org* <http://blog.afkham.org>* >>>>>>>>>>> twitter: >>>>>>>>>>> **http://twitter.com/afkham_azeez*<http://twitter.com/afkham_azeez> >>>>>>>>>>> * >>>>>>>>>>> linked-in: **http://lk.linkedin.com/in/afkhamazeez* >>>>>>>>>>> * >>>>>>>>>>> * >>>>>>>>>>> *Lean . Enterprise . Middleware* >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> _______________________________________________ >>>>>>>>>>> Dev mailing list >>>>>>>>>>> [email protected] >>>>>>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> ============================ >>>>>>>>>> Srinath Perera, Ph.D. >>>>>>>>>> http://www.cs.indiana.edu/~hperera/ >>>>>>>>>> http://srinathsview.blogspot.com/ >>>>>>>>>> >>>>>>>>>> _______________________________________________ >>>>>>>>>> Dev mailing list >>>>>>>>>> [email protected] >>>>>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> *S. Suhothayan >>>>>>>>> * >>>>>>>>> Software Engineer, >>>>>>>>> Data Technologies Team, >>>>>>>>> *WSO2, Inc. **http://wso2.com >>>>>>>>> <http://wso2.com/>* >>>>>>>>> *lean.enterprise.middleware.* >>>>>>>>> >>>>>>>>> *email: **[email protected]* <[email protected]>* cell: (+94) 779 756 757 >>>>>>>>> blog: >>>>>>>>> **http://suhothayan.blogspot.com/*<http://suhothayan.blogspot.com/> >>>>>>>>> * >>>>>>>>> twitter: >>>>>>>>> **http://twitter.com/suhothayan*<http://twitter.com/suhothayan> >>>>>>>>> * >>>>>>>>> linked-in: **http://lk.linkedin.com/in/suhothayan* >>>>>>>>> * >>>>>>>>> * >>>>>>>>> >>>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> Dev mailing list >>>>>>>>> [email protected] >>>>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Regards, >>>>>>> >>>>>>> Tharindu >>>>>>> >>>>>>> blog: http://mackiemathew.com/ >>>>>>> M: +94777759908 >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> ============================ >>>>>> Srinath Perera, Ph.D. >>>>>> Senior Software Architect, WSO2 Inc. >>>>>> Visiting Faculty, University of Moratuwa >>>>>> Member, Apache Software Foundation >>>>>> Research Scientist, Lanka Software Foundation >>>>>> Blog: http://srinathsview.blogspot.com/ >>>>>> Photos: http://www.flickr.com/photos/hemapani/ >>>>>> Phone: 0772360902 >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> *Afkham Azeez* >>>>> Director of Architecture; WSO2, Inc.; http://wso2.com >>>>> Member; Apache Software Foundation; http://www.apache.org/ >>>>> * <http://www.apache.org/>** >>>>> email: **[email protected]* <[email protected]>* cell: +94 77 3320919 >>>>> blog: **http://blog.afkham.org* <http://blog.afkham.org>* >>>>> twitter: >>>>> **http://twitter.com/afkham_azeez*<http://twitter.com/afkham_azeez> >>>>> * >>>>> linked-in: **http://lk.linkedin.com/in/afkhamazeez* >>>>> * >>>>> * >>>>> *Lean . Enterprise . Middleware* >>>>> >>>>> >>>> >>> >>> >>> -- >>> Regards, >>> >>> Tharindu >>> >>> blog: http://mackiemathew.com/ >>> M: +94777759908 >>> >>> >> >> >> -- >> *Afkham Azeez* >> Director of Architecture; WSO2, Inc.; http://wso2.com >> Member; Apache Software Foundation; http://www.apache.org/ >> * <http://www.apache.org/>** >> email: **[email protected]* <[email protected]>* cell: +94 77 3320919 >> blog: **http://blog.afkham.org* <http://blog.afkham.org>* >> twitter: **http://twitter.com/afkham_azeez*<http://twitter.com/afkham_azeez> >> * >> linked-in: **http://lk.linkedin.com/in/afkhamazeez* >> * >> * >> *Lean . Enterprise . Middleware* >> >> > > > -- > Regards, > > Tharindu > > blog: http://mackiemathew.com/ > M: +94777759908 > > -- *Afkham Azeez* Director of Architecture; WSO2, Inc.; http://wso2.com Member; Apache Software Foundation; http://www.apache.org/ * <http://www.apache.org/>** email: **[email protected]* <[email protected]>* cell: +94 77 3320919 blog: **http://blog.afkham.org* <http://blog.afkham.org>* twitter: **http://twitter.com/afkham_azeez*<http://twitter.com/afkham_azeez> * linked-in: **http://lk.linkedin.com/in/afkhamazeez* * * *Lean . Enterprise . Middleware*
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
