Hi Currently the concept of current event & expired events live within the query and all events going out to a stream are converted back to current events. So its hard for the application to keep track of the window and aggregation states like count, avg, std, etc... Further window implementations can very based on its implementations hence in some cases knowing what events entered and existed will not be enough to recreate the window during failure.
The recommended way to keep track of state in Siddhi is via snapshots, you can take snapshots of the siddhi Runtime with a reasonable time frame. and also buffer a copy of the events sent to siddhi after that snapshot, with this method when there is a failure we should restore the latest snapshot and replay the events which are sent after the last snapshot. The tricky part is the way you buffer events after snapshot, using Kafka and replaying is one option. Regards Suho On Thu, Nov 26, 2015 at 10:01 AM, Edward Zhang <[email protected]> wrote: > I tried expired events before, it only works for the query without > groupby. If the query contains groupby and having clause, then it only emit > just expired event when having conditions is satisfied. > > For example > > String cseEventStream = "define stream TempStream (user string, cmd string);"; > String query = "@info(name = 'query1') from TempStream#window.length(4) " > + " select user, cmd, count(user) as cnt " + > " group by user " + > "having cnt > 3 " > + " insert all events into DelayedTempStream"; > > > If we send events as follows, it will not generate expired events at all. > > inputHandler.send(new Object[]{"user", "open1"}); > inputHandler.send(new Object[]{"user", "open2"}); > inputHandler.send(new Object[]{"user", "open3"}); > inputHandler.send(new Object[]{"user", "open4"}); > inputHandler.send(new Object[]{"user", "open5"}); > > > > Thanks > Edward Zhang > > On Wed, Nov 25, 2015 at 6:50 PM, Srinath Perera <[email protected]> wrote: > >> Adding Suho >> >> Hi Edward, >> >> Each window give you a stream of expired events as well. Would that work? >> >> >> https://docs.wso2.com/display/CEP400/SiddhiQL+Guide+3.0#SiddhiQLGuide3.0-Window >> >> Thank >> Srinath >> >> On Thu, Nov 19, 2015 at 5:37 AM, Edward Zhang <[email protected]> >> wrote: >> >>> Hi Siddhi team, >>> >>> Do we have anyway of tracking what events are removed from any type of >>> windows, length(batch), or time(batch)? I investigated that removeEvents >>> may not be the right solution. >>> >>> We have one requirement of migrating policy from one machine to another >>> machine but keeping internal state there. >>> >>> Eagle uses policy in storm infrastructure, but one machine which holds >>> the policy fails, then already-populated events in the window also are >>> gone. Sometimes it is bad especially when we have built up a long window >>> like monthly data. >>> >>> One possible way is to keep all events in the siddhi window to be >>> snapshotted into application domain. Another way is to keep tracking what >>> events are coming in and out, so application can track what are left in >>> siddhi window. >>> >>> Here is the ticket for Eagle >>> https://issues.apache.org/jira/browse/EAGLE-39 >>> >>> Do you have similar request before? Or what do you suggest? >>> >>> Thanks >>> Edward Zhang >>> >>> _______________________________________________ >>> Dev mailing list >>> [email protected] >>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>> >>> >> >> >> -- >> ============================ >> Srinath Perera, Ph.D. >> http://people.apache.org/~hemapani/ >> http://srinathsview.blogspot.com/ >> > > -- *S. Suhothayan* Technical Lead & Team Lead of WSO2 Complex Event Processor *WSO2 Inc. *http://wso2.com * <http://wso2.com/>* lean . enterprise . middleware *cell: (+94) 779 756 757 | blog: http://suhothayan.blogspot.com/ <http://suhothayan.blogspot.com/>twitter: http://twitter.com/suhothayan <http://twitter.com/suhothayan> | linked-in: http://lk.linkedin.com/in/suhothayan <http://lk.linkedin.com/in/suhothayan>*
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
