Hi Sriskandarajah, Thanks for the quick response.
When you are saying scenario 1 and 2, which scenarios do you refer to? In option 2 : 1) When you are writing "a1.source_ip and a2.destination_ip" without operator, Is it like the group by feature I described? 2) Is the requirement of "event will stay open for 10 minutes" supported? 3) Is the "collect" aggregation function supported? In option 1: 1) How the requirement of uniqueness of port is supported? 2) Questions 2 & 3 as above 3) Aren't we missing having(count) >=20? When is the next release planned? I'll post new issues on the support page in the future. Thanks a lot. From: Sriskandarajah Suhothayan [mailto:[email protected]] Sent: Monday, August 12, 2013 11:23 AM To: Elran Dvir Cc: Lasantha Fernando; [email protected] Subject: Re: [Dev] Is my use case suuported in WSO2 CEP (Siddhi)? There is two ways to solve the 1st scenario 1. write an window extension by improving the time window to only send the unique events E.g from connectionStream.window:ext:uniquetime(source_ip,destination_ip,1 min) insert into resultStream count (port) as event count source_ip a2.destination_ip as destination_ip 2. write a pattern E.g from every a1=connectionStream -> a2=connectionStream[a1.source_ip and a2.destination_ip and a1.port!=port] -> a3=[a1.source_ip and a2.destination_ip and a1.port!=port and a2.port!=port] ... (write up to a20 :( ) within 1 min insert into resultStream a1.source_ip as source_ip a2.destination_ip as destination_ip but Scenario 2 is not supported out of the box by siddhi now. With the CEP 3.0.0 release this will be possible as it has output rate limiting functionality I have also created a jira for uniquetime window since its looks promising. We'll try to add that to next release.[1] I advice you to go through to WSO2 Support page[2] for details on Evaluation support and other supports through this we can provide support not only by answering your technical questions but also by writing custom functions for Siddhi Regards Suho [1]https://wso2.org/jira/browse/CEP-322 [2]http://wso2.com/support On Sun, Aug 11, 2013 at 5:16 PM, Elran Dvir <[email protected]<mailto:[email protected]>> wrote: Hi Lasantha, Thanks for the quick response. I have already read the documentation, but it's not clear to me how I can combine all the features to support my use case. If it's not a bother, it will be much appreciated if you can write my example in Siddhi query syntax. Thank you very much. From: Lasantha Fernando [mailto:[email protected]<mailto:[email protected]>] Sent: Sunday, August 11, 2013 2:34 PM To: Elran Dvir Cc: [email protected]<mailto:[email protected]> Subject: Re: [Dev] Is my use case suuported in WSO2 CEP (Siddhi)? Hi Eiran, You can always ask 'User Mailing List' type questions at StackOverflow. For your query, yes , the scenario you described can be implemented with WSO2 CEP (Siddhi). You can look at our documentation in [1]. >From what I understood of your query, I think what you are describing would be >a typical usecase for WSO2 CEP. * Creating a new event based on a set of multiple events maybe done with counting patterns [2]. * For do it for a period of T1, you can use time windows [3] in Siddhi. * The 'At least X events' scenario is also supported in Siddhi sequences and patterns. * Aggregate functions/ Unique events are also supported in Siddhi. * Keeping the new event open for T2 time is also possible by sending the new event to a new stream and applying a window of time T2 for that. * Getting an update of that T2 window on T3 time intervals might not be possible in CEP 2.1.0. However, this exact feature is implemented with CEP 3.0.0 which is the next release. The alpha pack for CEP 3.0.0 will be available within the next two weeks. (The functionality is already their in the codebase with Output Rate Limiting feature). Basically, I think all your requirements can be done from WSO2 CEP out of the box. Additionally, Siddhi allows you to write custom extensions for aggregators, windows, transformers, conditions, expressions [4] which allows more flexibility in implementing a specific usecase. If the "collect" aggregate function requires some custom behaviour, you can write an output processor extension to implement this functionality. HTH, Lasantha [1] http://docs.wso2.org/wiki/display/CEP210/Complex+Event+Processor+Documentation [2] http://docs.wso2.org/wiki/display/CEP210/Patterns [3] http://docs.wso2.org/wiki/display/CEP210/Windows [4] http://docs.wso2.org/wiki/display/CEP210/Writing+Extentions+to+Siddhi On 11 August 2013 13:56, Elran Dvir <[email protected]<mailto:[email protected]>> wrote: Hi all, I sent this question to the users' mailing list, but I think there was a problem with the subscription, so I am trying it also here. I am checking different types of CEP engines to embed inside our java application. I am new to WSO2 CEP and I'm trying to understand whether the following use case is supported - any help on the following will be greatly appreciated: I would like to create a new event based on multiple events (all of the same type meeting a set of conditions) occurring over a given period of time T1. For each combination of values for fieldA and fieldB, a new group of event candidates should be opened (fieldA and fieldB are group by fields. Each combination of values of these fields, should be treated separately). The event should be created when at least X events occurred over the period. Count the events based on unique values of fieldC and fieldD (for a given combination of fieldA and fieldB, if you notice an event with already existing values of the combination of fieldC and fieldD, it should not be counted). If all conditions described above are met, create the desired new event. The new event will stay open for duration of T2, and update will be sent for it every T3. Aside from the above, I need an aggregation function (besides count) of "collect" : in the new event the value of fieldE will be the collection of (preferably distinct) values of fieldE in originating events. Example: Port scan event - the basic event is connection. For each combination of source_ip and destination_ip (group by fields), detect a port scan event if over a minute (T1) there more than 20 (X) events with different ports (unique field). The event will stay open for 10 minutes (T2) and an update will be sent every 1 minute (T3). Every update will contain the count of events, source_ip, destination_ip and collection of services. Thanks a lot. _______________________________________________ Dev mailing list [email protected]<mailto:[email protected]> http://wso2.org/cgi-bin/mailman/listinfo/dev -- Lasantha Fernando Software Engineer - Data Technologies Team WSO2 Inc. http://wso2.com email: [email protected]<mailto:[email protected]> mobile: (+94) 71 5247551<tel:%28%2B94%29%2071%205247551> Email secured by Check Point _______________________________________________ Dev mailing list [email protected]<mailto:[email protected]> http://wso2.org/cgi-bin/mailman/listinfo/dev -- S. Suhothayan Associate Technical Lead, WSO2 Inc. http://wso2.com <http://wso2.com/> lean . enterprise . middleware cell: (+94) 779 756 757 | blog: http://suhothayan.blogspot.com/ twitter: http://twitter.com/suhothayan | linked-in: http://lk.linkedin.com/in/suhothayan Email secured by Check Point
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
