On Fri, Jul 20, 2018 at 8:05 AM, Isuru Haththotuwa <[email protected]> wrote:
> > > On Fri, Jul 20, 2018 at 12:43 AM, Nuwan Dias <[email protected]> wrote: > >> What happens here is that real time analytics on SP is done based on time >> batch windows and sliding windows. The query on the events in these windows >> are based on the time of the window and not really on the time the actual >> event occurred (the time the request was processed on API Manager). If we >> can find a way to solve the original problem of the data being queried >> based on the actual time of its occurrence, this problem would go away. >> > What if we use the sliding/batch window but group the events received by > the actual request time? > >> >> Although we argue that this problem exists only for the MG due to its >> aggregate and upload architecture, the same issue can occur for any other >> client that is sending data to SP after queueing on its side. The >> occurrence of the problem you are explaining can be very frequent for the >> MG due to its architecture. But really, the actual problem is something >> that exists for any client that queues and sends data to SP. I think we >> should try and solve that problem first. >> >> And also, one more thing we should do is to send data to SP from the MG >> directly when the connection is available. And only write to file when a >> connection doesn't exist. >> >> We might not be able to fully remove buffering of events (either in a > file, or in memory) in the usecase of GW and Analytics server. And to stop > real time alerting based on buffered data, we can include a condition in > the published events (ex.: isBuffered = true), which can be checked from > the SP side. This will work regardless of whether we are using MG or the > standard Gateway. > If we solve the summarization based on the original event time, we will be able to generate alert accurately. But we may not be able to send alert to users real time always. I think it is fine and at least we have alert history with correct values. > >> On Thu, Jul 19, 2018 at 11:58 AM Fazlan Nazeem <[email protected]> wrote: >> >>> Hi all, >>> >>> With Micorgateway Analytics support there is now the option to use the >>> same Analytics Server(SP for the upcoming release) for both Synapse based >>> gateway and Microgateway. With this being the most possible case, we will >>> have to redesign how the alerts will be working. The reason being, for >>> micro-gateway we are collecting analytics data into files, and then >>> uploading those in batches in predefined time intervals. >>> >>> This will result in events coming into the analytics server in >>> batches(not in real time). Generating alerts for these events will make no >>> sense, as these events are inherently not real-time. >>> >>> Therefore can we all agree on this fact, and stop supporting the alerts >>> feature for micro-gateways? >>> >>> >>> Thanks & Regards, >>> >>> *Fazlan Nazeem* >>> Senior Software Engineer >>> WSO2 Inc >>> Mobile : +94772338839 >>> [email protected] >>> >> >> >> -- >> Nuwan Dias >> >> Director - WSO2, Inc. http://wso2.com >> email : [email protected] >> Phone : +94 777 775 729 >> > > > > -- > Thanks and Regards, > > Isuru H. > +94 716 358 048* <http://wso2.com/>* > > > > _______________________________________________ > Architecture mailing list > [email protected] > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > > -- Rukshan Chathuranga. Software Engineer. WSO2, Inc. +94711822074
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
