On Tue, Jul 16, 2013 at 11:47 AM, Shariq Muhammed <[email protected]> wrote:
> Hi, > > > On Fri, Jul 12, 2013 at 12:01 PM, Amila Maha Arachchi <[email protected]>wrote: > >> Hi Shariq, >> >> We definitely need to publish the stats to the ST's keyspace. In the >> aPaaS, user does not have any control over stat publishing since he/she >> does not see the AS UI. We, as the aPaaS hosting party, need to publish >> all the stats and present them to the user (there is no choice to the user >> in this case). >> >> Second thing, we need a usage agent component which can be used by any >> other components. Basically, this agent should be publishing to ST's >> keyspace and these stats are for aPaaS vendor's purposes (for showing >> the stats, billing etc.). Not only usage stats, there are other statswhich >> needs to be gathered. One example is, AF itself needs to capture events >> such as builds triggered, promote/demote etc. So, AF also should be able >> to use this usage.agent to publish data (after that its up to them to >> use toolboxes and do whatever analysis they want to). >> >> Regarding your original concern, yes we should be able to use this stat >> publishing component, but we need to publish data to ST. Thats clear and >> we know that by our SLive experience. In the aPaaS deployment, there is >> not stat publishing per tenant. Everything should go to the same place. >> >> Current usage agent cannot be used to publish any usage event (in the >> way it is written). Thats why we need to improve it. My idea is to have >> simple >> API like follows. >> >> publish (Event event, String streamDef) <-- Just an idea. Need to sit >> and think a little bit more >> > > I've been thinking about this and for me it seems like it'll be an > additional wrapper for the existing data-bridge API. Given that the main > stat collecting bundle should also define the event stream, etc., I don't > see any major advantages of delegating the "publish" functionality when we > already use data-bridge for the same purpose. IMHO it's much cleaner to > have a single bundle to collect the stats, create the stream def and > publish it. We already have such publisher for API-M, mediation stuff, > service stats etc ... > Yes. I too had second thoughts after having some offline discussions with several folks. It seems we cannot provide additional advantage than the data-bridge agent. So, it seems each product will have to publish their stats accordingly. But, if one product has 3-4 components which needs to publish data to BAM, then it will be a good idea for them to have their own component which handles that (for example AF will have several components which will publishing to BAM). Because there some components which should not handle such publishing (which can affect its performance), rather they should delegate them. >> For example AF can publish to org,wso2.carbon.appfactory stream def. AS >> can publish to org.wso2.carbon.statistics or something similar. >> >> Any components should be able to get hold of usage agent as an OSGi >> service and call above method. Then usage agent should store it in a map >> and publish periodically (once in every 2 seconds etc.. as we do at the >> moment). This waym the usage agent will become very simpler and usable >> than the existing one. >> >> WDYT, >> >> Regards, >> AmilaM. >> >> >> On Fri, Jul 12, 2013 at 11:26 AM, Shariq Muhammed <[email protected]>wrote: >> >>> Hi folks, >>> >>> I was looking into capturing webapp statistics for aPaaS work, and came >>> across the webapp.stat.publisher component written by the AS team. It >>> uses a tomcat valve [1] and publish a whole heap of stats, the list can be >>> found in the class at [2]. >>> >>> The new stat component captures way more data than the usage components >>> we had before, only missing piece as of now is the request/response >>> bandwidth. What we've done now is to patch tomcat RequestInfo.java class, >>> if we can somehow capture the bandwidth also in the valve we can get rid of >>> tomcat.patch, tomcat.fragment.dummy bundles and a few classes from >>> tomcat.ext bundle in the kernel. >>> >>> By looking into the tomcat code, it seems we can get the coyote.Request >>> (object in RequestInfo) within the valve, but the bandwidth values >>> returned are 0. >>> >>> request.getCoyoteRequest().getBytesRead(); // => 0 >>> request.getCoyoteRequest().getResponse().getContentWritten() // => 0 >>> >>> But the correct values are returned in the patched RequestInfo class, >>> so I am still trying to figure out what could be the issue. If we can >>> capture this value also at the value we can publish all properties from one >>> place. >>> >>> So first of all I would like to know if it's fine to go ahead with this >>> implementation and improve it to cater aPaaS requirements or do we need >>> a separate bundle (would be a fork of this one in the end). IMO we should >>> use this component and improve it since there is no point having 2 bundles >>> doing the same thing. >>> >>> One concern, the component is designed for per tenant publishing, and >>> the tenant chooses whether or not to publish and where to publish. If we >>> are to capture stats centrally we need to publish to a ST CF regardless >>> of tenant enable or disable statistics, can be done using a config option. >>> >>> >>> @Geeth - Noticed that even after a tenant enable stat publishing for >>> webapps, we need to again enable publishing per webapp. IMO the model >>> should be that if a tenant enable publishing, stats for all deployed >>> webapps should be published and tenant should be given the option to >>> disable stats selectively. >>> >>> Thoughts ... ?! >>> >>> [1] - >>> https://svn.wso2.org/repos/wso2/carbon/platform/trunk/components/data-agents/org.wso2.carbon.bam.webapp.stat.publisher/src/main/java/org/wso2/carbon/bam/webapp/stat/publisher/WebAppStatisticPublisherValve.java >>> [2] - >>> https://svn.wso2.org/repos/wso2/carbon/platform/trunk/components/data-agents/org.wso2.carbon.bam.webapp.stat.publisher/src/main/java/org/wso2/carbon/bam/webapp/stat/publisher/data/WebappStatEventData.java >>> >>> -- >>> Thanks, >>> Shariq. >>> Phone: +94 777 202 225 >>> >> >> >> >> -- >> *Amila Maharachchi* >> Senior Technical Lead >> WSO2, Inc.; http://wso2.com >> >> Blog: http://maharachchi.blogspot.com >> Mobile: +94719371446 >> >> > > > -- > Thanks, > Shariq. > Phone: +94 777 202 225 > -- *Amila Maharachchi* Senior Technical Lead WSO2, Inc.; http://wso2.com Blog: http://maharachchi.blogspot.com Mobile: +94719371446
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
