Hi All, Looks like we have few good suggestions.. So lets finalize a optimum set of data and move on please.. :) Geeth could you followup Kasun G's work and send a finalized list?.
thanks, On Sun, May 12, 2013 at 11:11 PM, Sriskandarajah Suhothayan <[email protected]>wrote: > > > > On Sun, May 12, 2013 at 6:44 PM, Kasun Weranga <[email protected]> wrote: > >> >> >> >> On Sat, May 11, 2013 at 1:02 PM, Amila Suriarachchi <[email protected]>wrote: >> >>> >>> >>> >>> On Fri, May 10, 2013 at 9:21 PM, Sagara Gunathunga <[email protected]>wrote: >>> >>>> >>>> >>>> >>>> On Fri, May 10, 2013 at 11:10 PM, Kasun Weranga <[email protected]>wrote: >>>> >>>>> >>>>> >>>>> >>>>> On Thu, May 9, 2013 at 12:05 PM, Kasun Gajasinghe <[email protected]>wrote: >>>>> >>>>>> Hi Srinath, >>>>>> >>>>>> On Thu, May 9, 2013 at 9:21 AM, Srinath Perera <[email protected]>wrote: >>>>>> >>>>>>> Kasun, one thing I notice is that payload data are static. Basically >>>>>>> when we have webappName and resource path, we can look up everything >>>>>>> else. >>>>>>> (well queustion is where, but that we should solve seperately). >>>>>>> >>>>>> >>>>>> Yes, this makes sense. >>>>>> >>>>>> >>>>>>> >>>>>>> I think we should not repeat that that information (e.g. dispaly >>>>>>> name etc) for every event. >>>>>>> >>>>>>> >>>>>> +1 >>>>>> >>>>>> >>>>>>> Also I think we need information like how long the request took and >>>>>>> when do we received the request, details if there was any error. >>>>>>> >>>>>>> >>>>>> We can measure the time taken in the Valve. It will include the >>>>>> network delay as well, but that's OK I guess. It will not include the >>>>>> time >>>>>> taken by the valves that get executed before the WebappStatValve though. >>>>>> >>>>>> We can determine if there is an error using the HTTP status code. We >>>>>> can send faulty details if the response status code starts with 4xx or >>>>>> 5xx. >>>>>> >>>>> >>>>> Even-though status code reflects the success or failure of the >>>>> invocation. In addition to status code it is better if we can publish >>>>> the request_count, response_count, fault_count for that invocation, like >>>>> in >>>>> service stats agent. (ex:- for successful invocation request_count =1, >>>>> response_count=1, fault_count=0) >>>>> >>>> >>>> Here in this thread let's focus on page based web applications not >>>> JAX-WS/JAX-RS ( For them requirements are identical to Axis2 services) then >>>> we don't have count for request/response/faults instead we can provide >>>> following numbers. >>>> >>>> 1. AVG response time for any page >>>> 2. AVG # visit on each page >>>> 3. AVG time visit on each page >>>> 4. # sessions over the time >>>> 5. maximum and avg length of a session >>>> 6. Client data such as IP, Domain >>>> >>> >>> Those stats can be calculated at the analyzer level. Here we need to >>> think about the data to be send to the BAM. IIRC BAM publishes data per >>> each request. >>> >> >> Yes above data should be calculated at analyzer level. What we need to do >> is publish useful data at each invocation which can be used to calculate >> above values in BAM side. Also by publishing aggregated information we >> might miss some important data specially for real time monitoring using CEP >> (ex:- for pattern matching). >> >> +1, adding to Kasuns surgeons, AVG need to be configurable and E.g 1 > hour/day/year and > there might also be various other combinations users need to monitor and > notify. > > hence sending per message specific data will be the optimal > > > Suho > > Thanks, >> KasunW. >> >>> >>> thanks, >>> Amila. >>> >>>> >>>> I think it's better to look at analytic tools like Google analytic to >>>> find more options. >>>> >>>> Thanks ! >>>> >>>> >>>>> >>>>> Because If users want to write their own analytics using these >>>>> statistics it will easy their task. (CEP queries and BAM hive scripts will >>>>> be much simpler) >>>>> >>>>> Thanks, >>>>> KasunW. >>>>> >>>>>> >>>>>> In addition, we can grab some more information from the Tomcat >>>>>> AccessLogValve too [1]. This valve is used for logging purposes. >>>>>> >>>>>> [1] >>>>>> http://tomcat.apache.org/tomcat-6.0-doc/api/org/apache/catalina/valves/AccessLogValve.html >>>>>> >>>>>> >>>>>>> --Srinath >>>>>>> >>>>>>> >>>>>>> On Wed, May 8, 2013 at 5:53 PM, Kasun Gajasinghe <[email protected]>wrote: >>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> I'm trying to finalize the webapp request stats data that AS should >>>>>>>> send to BAM. I have come with the following list of stats to be sent >>>>>>>> to BAM >>>>>>>> as part of the Event Data. These are categorized in to payload data and >>>>>>>> metadata. Please share your suggestions on these, and what other data >>>>>>>> we >>>>>>>> can send. >>>>>>>> >>>>>>>> We can also make it to send the request message body, and response >>>>>>>> message body but it would add a significant overhead. >>>>>>>> >>>>>>>> Currently, webapp developers need to add a a context-param called >>>>>>>> 'enable.statistics' to the web.xml in webapps. >>>>>>>> >>>>>>>> <context-param> >>>>>>>> <param-name>*enable.statistics*</param-name> >>>>>>>> <param-value>true</param-value> >>>>>>>> </context-param> >>>>>>>> >>>>>>>> They can configure the webapp data agent configuration and BAM >>>>>>>> credentials via the management console. The configuration is saved in >>>>>>>> the >>>>>>>> registry of the given tenant. Each tenant has to configure the data >>>>>>>> agent >>>>>>>> this way. This behavior is quite like the Service Data Agent >>>>>>>> configuration. >>>>>>>> >>>>>>>> >>>>>>>> streamId='bam_webapp_statistics_publisher:3.0.0', >>>>>>>> name='bam_webapp_statistics_publisher', >>>>>>>> version='1.0.0', >>>>>>>> nickName='WebappDataAgent', >>>>>>>> description='Publish webapp statistics events', >>>>>>>> >>>>>>>> Payload data - >>>>>>>> >>>>>>>> webappName >>>>>>>> webappDisplayName - The <display-name> set in web.xml >>>>>>>> webappOwnerTenant >>>>>>>> webappVersion - web-app servlet version. Ex. 2.5, 3.0 >>>>>>>> timestamp >>>>>>>> webappContext - ex. /t/example.com/jaxwebapps/jaxrs_basic >>>>>>>> resourcePath - ex. For a request url of >>>>>>>> http://localhost:9763/t/example.com/jaxwebapps/jaxrs_basic/services/customers/customerservice/customers/123, >>>>>>>> the resource path will be /customers/customerservice/customers/123 >>>>>>>> webappType - Possible values are JAX-WS/JAX-RS/Generic >>>>>>>> >>>>>>>> >>>>>>>> Metadata - >>>>>>>> >>>>>>>> >>>>>>>> httpMethod - The HTTP method of the request >>>>>>>> contentType - Content Type of the request >>>>>>>> responseContentType - Content Type of the response >>>>>>>> responseHttpStatusCode - HTTP status code of the response >>>>>>>> userAgent - client user-agent >>>>>>>> remoteAddress - client address >>>>>>>> referer - HTTP referer header >>>>>>>> authType - Tomcat level authentication, if set. ex. >>>>>>>> BASIC_AUTH. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> KasunG >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> Architecture mailing list >>>>>>>> [email protected] >>>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> ============================ >>>>>>> Srinath Perera, Ph.D. >>>>>>> http://www.cs.indiana.edu/~hperera/ >>>>>>> http://srinathsview.blogspot.com/ >>>>>>> >>>>>>> _______________________________________________ >>>>>>> Architecture mailing list >>>>>>> [email protected] >>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> *Kasun Gajasinghe* >>>>>> Software Engineer; >>>>>> Development Technologies Team, WSO2 Inc.; http://wso2.com >>>>>> >>>>>> , >>>>>> *email: **kasung AT spamfree wso2.com >>>>>> >>>>>> ** cell: **+94 (77) 678-0813* >>>>>> *linked-in: *http://lk.linkedin.com/in/gajasinghe >>>>>> >>>>>> * >>>>>> * >>>>>> *blog: **http://kasunbg.org* <http://kasunbg.org> >>>>>> >>>>>> * >>>>>> twitter: **http://twitter.com/kasunbg* <http://twitter.com/kasunbg> >>>>>> >>>>>> * >>>>>> * >>>>>> >>>>>> _______________________________________________ >>>>>> Architecture mailing list >>>>>> [email protected] >>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> *Kasun Weranga* >>>>> ** >>>>> Member, Management Committee - Data Technologies >>>>> Software Engineer >>>>> *WSO2, Inc. >>>>> *lean.enterprise.middleware. >>>>> mobile : +94 772314602 >>>>> <http://sanjeewamalalgoda.blogspot.com/>blog : >>>>> http://kasunweranga.blogspot.com/ >>>>> >>>>> _______________________________________________ >>>>> Architecture mailing list >>>>> [email protected] >>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>> >>>>> >>>> >>>> >>>> -- >>>> Sagara Gunathunga >>>> >>>> Technical Lead; WSO2, Inc.; http://wso2.com >>>> V.P Apache Web Services; http://ws.apache.org/ >>>> Linkedin; http://www.linkedin.com/in/ssagara >>>> Blog ; http://ssagara.blogspot.com >>>> >>>> >>>> _______________________________________________ >>>> Architecture mailing list >>>> [email protected] >>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>> >>>> >>> >>> >>> -- >>> *Amila Suriarachchi* >>> >>> Software Architect >>> WSO2 Inc. ; http://wso2.com >>> lean . enterprise . middleware >>> >>> phone : +94 71 3082805 >>> >>> _______________________________________________ >>> Architecture mailing list >>> [email protected] >>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>> >>> >> >> >> -- >> *Kasun Weranga* >> ** >> Member, Management Committee - Data Technologies >> Software Engineer >> *WSO2, Inc. >> *lean.enterprise.middleware. >> mobile : +94 772314602 >> <http://sanjeewamalalgoda.blogspot.com/>blog : >> http://kasunweranga.blogspot.com/ >> >> _______________________________________________ >> Architecture mailing list >> [email protected] >> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >> >> > > > -- > *S. Suhothayan > * > *Software Engineer, > Member, Management Committee - Data Technologies Team, > * > * * > *WSO2 Inc. **http://wso2.com > <http://wso2.com/>* > *lean . enterprise . middleware* > > *cell: (+94) 779 756 757 > blog: **http://suhothayan.blogspot.com/* <http://suhothayan.blogspot.com/> > * > twitter: **http://twitter.com/suhothayan* <http://twitter.com/suhothayan>* > linked-in: **http://lk.linkedin.com/in/suhothayan* > * > * > > _______________________________________________ > Architecture mailing list > [email protected] > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > > -- Supun Malinga, Software Engineer, WSO2 Inc. http://wso2.com http://wso2.org email - [email protected] <[email protected]> mobile - 071 56 91 321
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
