Hi Jay,
Since you are maintaining a jobTable, you can use that to identify
completed and non-completed jobs in a given time frame. Refer to the below
snippet;
@Plan:name('ExecutionPlan')
> @Import('subStream:1.0.0')
> define stream subStream (pid int, sid int, status string);
> @Import('pubStream:1.0.0')
> define stream pubStream (pid int, status string);
> @Export('jobsNotCompletedInTime:1.0.0')
> define stream jobsNotCompletedInTime (pid int, sid int);
> @Export('jobsCompletedInTime:1.0.0')
> define stream jobsCompletedInTime (pid int, sid int);
> @Export('foo:1.0.0')
> define stream foo (pid int, sid int);
> @From(eventtable='rdbms', jdbc.url='jdbc:mysql://localhost:3306/pubsub',
> username='root', password='root', driver.name='com.mysql.jdbc.Driver',
> table.name='jobTable')
> define table jobTable(pid int, sid int);
> from pubStream as p join jobTable as t
> on p.pid == t.pid
> select p.pid, t.sid
> insert into allPSJobs;
-- this will emit an event after 10 sec once a publish event arrives
from pubStream#window.time(10 sec)
> select *
> insert expired events into *expiredPublisherEvents*;
-- if the pattern matches within 10 sec, that means the job completed
> within time
from every (a=allPSJobs) -> s=subStream[pid==a.pid and sid == a.sid]
> within 10 sec
> select s.pid, s.sid
> insert into *jobsCompletedInTime*;
> -- delete completed jobs from the table, then what's left in the table are
> the uncompleted jobs
> from jobsCompletedInTime
> delete jobTable
> on jobTable.pid == pid and jobTable.sid == sid;
> -- so if we join expiredPublisherEvents with the jobTable, the result
> will be the jobs that are not completed within time
from expiredPublisherEvents as ep join jobTable as t
> on ep.pid == t.pid
> select ep.pid, t.sid
> insert into *jobsNotCompletedInTime*;
> -- now delete those uncompleted jobs from the jobTable as well
> from jobsNotCompletedInTime
> delete jobTable
> on jobTable.pid == pid and jobTable.sid == sid;
Hope that helped;
Regards,
Grainier.
On Fri, Sep 29, 2017 at 11:17 AM, Jayesh Senjaliya <[email protected]>
wrote:
> Hi Gobinath,
>
> Thanks for the suggestion about absence pattern, but we wont be able to
> upgrade to Siddhi 4 anytime soon.
>
> I am basically at the point where I can get all the relevant (subscribe)
> events that happened during the given interval of first arrival of publish
> events.
>
> here AllPublisher = all registered publisher-subscriber events ( for each
> publisher event )
>
> from registeryTable as s join #publisher as p on p.pid == s.pid
> select p.pid, s.sid insert into AllPublisher;
>
>
> from every(a=AllPublisher) -> s=subscriber[pid == a.pid]
> within 1 sec
> select a.pid, s.sid
> insert into completed_jobs_in_1_sec;
>
> now, i need to find out those events from publisher (a) that dint had any
> match from s within 1 sec.
>
> which I was expecting to be available with " insert *expired events* into
> not_completed_jobs", but looks like expired events are only available when
> window is used. I am also looking at the code to see if i should add this.
>
> Thanks
> Jayesh
>
>
>
>
>
>
>
>
>
> On Wed, Sep 27, 2017 at 4:21 AM, Gobinath <[email protected]> wrote:
>
>> Hi,
>>
>> If you can use Siddhi 4 snapshot release, it can be done using the new
>> feature 'Absent Pattern' added to Siddhi 4. The query to detect the events
>> that do not match the condition within 10 seconds is given below:
>>
>> from every e1=publisher -> not subscriber[e1.pid == pid] for 10 sec
>> select e1.pid
>> insert into not_completed_jobs_in_time;
>>
>> The above query waits for 10 seconds from the arrival of every publisher
>> event and if there is no subscriber event with an id satisfying the
>> condition arrived within that waiting period, the id of the publisher event
>> will be inserted into the not_completed_jobs_in_time stream.
>>
>> I guess the official document for Siddhi 4 is under construction. So you
>> can find more details about absent pattern at [1]
>>
>> Still, Siddhi 4 is not production ready so I wonder whether you can use
>> this feature or not.
>>
>> [1] http://www.javahelps.com/2017/08/detect-absence-of-events-ws
>> o2-siddhi.html
>>
>>
>>
>>
>> On Tue, Sep 26, 2017 at 10:05 PM, Jayesh Senjaliya <[email protected]>
>> wrote:
>>
>>> Hi Grainier,
>>>
>>> ya, i came across that example page, but i think that does not work in
>>> my use-case which is as follow.
>>>
>>> i have a publish event followed by multiple subscribe event for the same
>>> publish job.
>>> now i want to catch if certain jobs (publish -> subscribe) has been
>>> finished with 10 sec.
>>> I have all the registered jobs in db table, which i use to gather all
>>> the required publish-subscribe job events.
>>>
>>> define table jobTable( pid string, sid string);
>>> define stream pubStream (pid int, status string);
>>> define stream subStream (pid int, sid int, status string);
>>>
>>> -- this will get all the publish-> subscribe jobs events as master list
>>> from pubStream as p join jobTable as t
>>> on p.pid == t.pid
>>> select p.pid, t.sid insert into allPSJobs;
>>>
>>> -- this is where i need to do intersection where if subStream event is
>>> seen within 2 sec then remove that from master list ( allPSJobs ) if not
>>> include that in not_completed_jobs_in_time
>>>
>>> from every ( a=allPSJobs ) -> s= subStream[sid == a.sid and pid==a.pid ]
>>> within 2 sec
>>> select s.pid, s.sid insert into completed_jobs_in_time;
>>>
>>>
>>> hope that make sense from what i am trying to do.
>>>
>>> Thanks
>>> Jayesh
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Mon, Sep 25, 2017 at 8:39 AM, Grainier Perera <[email protected]>
>>> wrote:
>>>
>>>> Hi Jay,
>>>>
>>>> You can try something similar to this to get non-matched events during
>>>> last 10 secs; You can find some documentation on this as well; link
>>>> <https://docs.wso2.com/display/CEP420/Sample+0111+-+Detecting+non-occurrences+with+Patterns>
>>>>
>>>>
>>>>
>>>>> define stream publisher (pid string, time string);
>>>>> define stream subscriber (pid string, sid string, time string);
>>>>
>>>>
>>>>> from publisher#window.time(10 sec)
>>>>> select *
>>>>> insert expired events into expired_publisher;
>>>>
>>>>
>>>>> from every pub=publisher -> sub=subscriber[pub.pid == pid] or
>>>>> exp=expired_publisher[pub.pid == pid]
>>>>> select pub.pid as pid, pub.time as time, sub.pid as subPid
>>>>> insert into filter_stream;
>>>>
>>>>
>>>>> from filter_stream [(subPid is null)]
>>>>> select pid, time
>>>>> insert into not_seen_in_last_10_sec_events;
>>>>
>>>>
>>>> Moreover, I didn't get what you meant by "also is there a way to
>>>> perform intersection of events based on grouping or time window ?" can you
>>>> please elaborate on this?
>>>>
>>>> Regards,
>>>>
>>>> On Mon, Sep 25, 2017 at 11:02 AM, Jayesh Senjaliya <[email protected]
>>>> > wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> is there a way to get events that didnt match within the given time
>>>>> frame.
>>>>>
>>>>> for example:
>>>>>
>>>>> define stream publisher (pid string, time string);
>>>>> define stream subscriber (pid string, sid string, time string);
>>>>>
>>>>> from every (e1=publisher) -> e2=subscriber[e1.pid == pid]
>>>>> within 10 sec
>>>>> select e1.pid, e2.sid
>>>>> insert into seen_in_last_10_sec_events;
>>>>>
>>>>>
>>>>> so if i have matching event above, i will see it in
>>>>> seen_in_last_10_sec_events, but is there a way to get all events or non
>>>>> matched events during that last 10 seconds from publisher or subscriber ?
>>>>>
>>>>> also is there a way to perform intersection of events based on
>>>>> grouping or time window ?
>>>>>
>>>>>
>>>>> Thanks
>>>>> Jay
>>>>>
>>>>> _______________________________________________
>>>>> Dev mailing list
>>>>> [email protected]
>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Grainier Perera
>>>> Senior Software Engineer
>>>> Mobile : +94716122384 <+94%2071%20612%202384>
>>>> WSO2 Inc. | http://wso2.com
>>>> lean.enterprise.middleware
>>>>
>>>
>>>
>>> _______________________________________________
>>> Dev mailing list
>>> [email protected]
>>> http://wso2.org/cgi-bin/mailman/listinfo/dev
>>>
>>>
>>
>>
>> --
>> *Gobinath** Loganathan*
>> Graduate Student,
>> Electrical and Computer Engineering,
>> Western University.
>> Email : [email protected]
>> Blog : javahelps.com <http://www.javahelps.com/>
>>
>>
>
>
--
Grainier Perera
Senior Software Engineer
Mobile : +94716122384
WSO2 Inc. | http://wso2.com
lean.enterprise.middleware
_______________________________________________
Dev mailing list
[email protected]
http://wso2.org/cgi-bin/mailman/listinfo/dev
