Hi Sinthuja, There is an ongoing effort to combine request, response and execution time event streams into a single stream and publish a single event instead of 3 events to the Stream processor. This is targeted for the Q3 release and can bring down the traffic by 3 times to the analytics server.
Mail Subject: Moving APIM Analytics to SP On Tue, Jun 19, 2018 at 5:05 PM Sinthuja Rajendran <[email protected]> wrote: > Hi, > > With my recent work with metrics and other monitoring systems, I'm > thinking whether our model of sending everything and calculate in the > analytics server side is correct. > > Basically IMHO, the majority of the product analytics use cases are > statistics calculation. For example, in APIM, we are calculating API > statistics, Subscription statistics, etc. And for this, we are sending > events for every request/response/fault to analytics server, and analytics > server is performing the actual statistics calculation on the events. > Therefore in the high traffic scenario, there are a lot of events needs to > be published to analytics server, and we are getting issues like "Event > Queue Full" in the gateway. > > My proposal is, what if we calculate the statistics on the local node > itself (similar to the edge analytics by running Siddhi within the > products), and only send the minute statistics summary to analytics server > to do the global (across all nodes) calculation. With this way, the traffic > to analytics server will be reduced drastically as only the final > calculated value for each group by combination will be reported. Therefore, > the analytics server can focus on global summarization and other global > monitoring aspects. > > Thanks, > Sinthuja. > > -- > *Sinthuja Rajendran* > Senior Technical Lead > WSO2, Inc.:http://wso2.com > > Blog: http://sinthu-rajan.blogspot.com/ > Mobile: +94774273955 > > > -- Thanks & Regards, *Fazlan Nazeem* Senior Software Engineer WSO2 Inc Mobile : +94772338839 [email protected]
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
