There is two ways to solve the 1st scenario 1. write an window extension by improving the time window to only send the unique events
E.g from connectionStream.window:ext:uniquetime(source_ip,destination_ip,1 min) insert into resultStream count (port) as event count source_ip a2.destination_ip as destination_ip 2. write a pattern E.g from every a1=connectionStream -> a2=connectionStream[a1.source_ip and a2.destination_ip and a1.port!=port] -> a3=[a1.source_ip and a2.destination_ip and a1.port!=port and a2.port!=port] ... *(write up to a20 :( )* within 1 min insert into resultStream a1.source_ip as source_ip a2.destination_ip as destination_ip but Scenario 2 is not supported out of the box by siddhi now. With the CEP 3.0.0 release this will be possible as it has output rate limiting functionality I have also created a jira for uniquetime window since its looks promising. We'll try to add that to next release.[1] I advice you to go through to WSO2 Support page[2] for details on Evaluation support and other supports through this we can provide support not only by answering your technical questions but also by writing custom functions for Siddhi Regards Suho [1]https://wso2.org/jira/browse/CEP-322 [2]http://wso2.com/support On Sun, Aug 11, 2013 at 5:16 PM, Elran Dvir <[email protected]> wrote: > Hi Lasantha, **** > > ** ** > > Thanks for the quick response.**** > > ** ** > > I have already read the documentation, but it’s not clear to me how I can > combine all the features to support my use case.**** > > If it’s not a bother, it will be much appreciated if you can write my > example in Siddhi query syntax.**** > > **** > > Thank you very much.**** > > ** ** > > *From:* Lasantha Fernando [mailto:[email protected]] > *Sent:* Sunday, August 11, 2013 2:34 PM > *To:* Elran Dvir > *Cc:* [email protected] > *Subject:* Re: [Dev] Is my use case suuported in WSO2 CEP (Siddhi)?**** > > ** ** > > Hi Eiran,**** > > ** ** > > You can always ask 'User Mailing List' type questions at StackOverflow.*** > * > > ** ** > > For your query, yes , the scenario you described can be implemented with > WSO2 CEP (Siddhi). You can look at our documentation in [1].**** > > ** ** > > From what I understood of your query, I think what you are describing > would be a typical usecase for WSO2 CEP.**** > > - Creating a new event based on a set of multiple events maybe done > with counting patterns [2]. **** > - For do it for a period of T1, you can use time windows [3] in Siddhi. > **** > - The 'At least X events' scenario is also supported in Siddhi > sequences and patterns.**** > - Aggregate functions/ Unique events are also supported in Siddhi.**** > - Keeping the new event open for T2 time is also possible by sending > the new event to a new stream and applying a window of time T2 for that. > **** > - Getting an update of that T2 window on T3 time intervals might not > be possible in CEP 2.1.0. However, this exact feature is implemented with > CEP 3.0.0 which is the next release. The alpha pack for CEP 3.0.0 will be > available within the next two weeks. (The functionality is already their in > the codebase with Output Rate Limiting feature).**** > > Basically, I think all your requirements can be done from WSO2 CEP out > of the box. Additionally, Siddhi allows you to write custom extensions for > aggregators, windows, transformers, conditions, expressions [4] which > allows more flexibility in implementing a specific usecase. If the > "collect" aggregate function requires some custom behaviour, you can write > an output processor extension to implement this functionality.**** > > ** ** > > HTH,**** > > ** ** > > Lasantha**** > > ** ** > > [1] > http://docs.wso2.org/wiki/display/CEP210/Complex+Event+Processor+Documentation > **** > > [2] http://docs.wso2.org/wiki/display/CEP210/Patterns > [3] http://docs.wso2.org/wiki/display/CEP210/Windows**** > > [4] http://docs.wso2.org/wiki/display/CEP210/Writing+Extentions+to+Siddhi* > *** > > ** ** > > On 11 August 2013 13:56, Elran Dvir <[email protected]> wrote:**** > > Hi all,**** > > **** > > I sent this question to the users’ mailing list, but I think there was a > problem with the subscription, so I am trying it also here.**** > > I am checking different types of CEP engines to embed inside our java > application. **** > > I am new to WSO2 CEP and I’m trying to understand whether the following > use case is supported – any help on the following will be greatly > appreciated:**** > > **** > > I would like to create a new event based on *multiple* events (all of the > same type meeting a set of conditions) occurring *over a given period of > time T1*.**** > > For each combination of values for fieldA and fieldB, a new group of event > candidates should be opened (fieldA and fieldB are *group by* fields. > Each combination of values of these fields, should be treated separately). > **** > > The event should be created when *at least X events* occurred over the > period. Count the events based on *unique* values of fieldC and fieldD > (for a given combination of fieldA and fieldB, if you notice an event with > already existing values of the combination of fieldC and fieldD, it should > not be counted).**** > > If all conditions described above are met, create the desired new event. *The > new event will stay open for duration of T2, and update will be sent for it > every T3.***** > > * ***** > > Aside from the above, I need an *aggregation function (besides count) of > “collect”* : in the new event the value of fieldE will be the collection > of (preferably distinct) values of fieldE in originating events*. ***** > > * ***** > > Example:**** > > Port scan event – the basic event is connection. For each combination of > source_ip and destination_ip (group by fields), detect a port scan event if > over a minute (T1) there more than 20 (X) events with different ports > (unique field).**** > > The event will stay open for 10 minutes (T2) and an update will be sent > every 1 minute (T3). Every update will contain the count of events, > source_ip, destination_ip and collection of services.**** > > * ***** > > Thanks a lot.**** > > **** > > > _______________________________________________ > Dev mailing list > [email protected] > http://wso2.org/cgi-bin/mailman/listinfo/dev**** > > > > **** > > ** ** > > -- **** > > *Lasantha Fernando***** > > Software Engineer - Data Technologies Team**** > > WSO2 Inc. http://wso2.com**** > > ** ** > > email: [email protected]**** > > mobile: (+94) 71 5247551**** > > > > Email secured by Check Point **** > > _______________________________________________ > Dev mailing list > [email protected] > http://wso2.org/cgi-bin/mailman/listinfo/dev > > -- *S. Suhothayan * Associate Technical Lead, *WSO2 Inc. *http://wso2.com * <http://wso2.com/>* lean . enterprise . middleware *cell: (+94) 779 756 757 | blog: http://suhothayan.blogspot.com/ twitter: http://twitter.com/suhothayan | linked-in: http://lk.linkedin.com/in/suhothayan* * *
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
