Its a bug! Here is the fix
Suho On Mon, Sep 30, 2013 at 8:23 AM, Srinath Perera <[email protected]> wrote: > OK sill output is wrong, basically it is fired twice with count as 1 and > 2. How can I fix that (see below). I have attached the code > > AutoscaleData:[Event{streamId='AutoscaleData', timeStamp=1380554424779, > data=[1, 400], type=new}] > > Actions:[Event{streamId='Actions', timeStamp=1380554424779, data=[400], > type=new}] > > AutoscaleData:[Event{streamId='AutoscaleData', timeStamp=1380554424781, > data=[2, 400], type=new}] > > ============1============= > > AutoscaleData:[Event{streamId='AutoscaleData', timeStamp=1380554424779, > data=[1, 500], type=new}] > > Actions:[Event{streamId='Actions', timeStamp=1380554424779, data=[500], > type=new}] > > AutoscaleData:[Event{streamId='AutoscaleData', timeStamp=1380554424781, > data=[2, 500], type=new}] > > ============2============= > > AutoscaleData:[Event{streamId='AutoscaleData', timeStamp=1380554424779, > data=[1, 600], type=new}] > > Actions:[Event{streamId='Actions', timeStamp=1380554424779, data=[600], > type=new}] > > AutoscaleData:[Event{streamId='AutoscaleData', timeStamp=1380554424781, > data=[2, 600], type=new}] > > log:[Event{streamId='tStream', timeStamp=1380554424779, data=[192.121.0.1, > qsb], type=new}] > > log:[Event{streamId='tStream', timeStamp=1380554424781, data=[192.121.0.2, > qsb], type=new}] > > > On Mon, Sep 30, 2013 at 5:46 PM, Srinath Perera <[email protected]> wrote: > >> Ah group by in my query is wrong. Will try snd get back. >> On 30 Sep 2013 17:35, "Srinath Perera" <[email protected]> wrote: >> >>> Suho .. pls call me if you need any clarification. >>> >>> --Srinath >>> >>> >>> On Mon, Sep 30, 2013 at 5:35 PM, Srinath Perera <[email protected]>wrote: >>> >>>> Hi Suho, >>>> >>>> Sorry, answer you gave does not work. Still count is wrong. We need to >>>> clean this up, but before that I need a query that handle the following >>>> scenario for stratos. Following is the scenario. >>>> >>>> When node comes up it send a up messages and when they goes down, it >>>> send a down message. I store them to an event table. >>>> >>>> LB send a event with number of connections. I join them with event >>>> table and send an action event if connections/number events > 300. >>>> >>>> How can I do that? I felt above handles it, but the problem is it >>>> count( ..) remembers all evaluations. How can I join the incoming event >>>> with event table, count how many results comes out, send one action event >>>> if counts is OK? >>>> >>>> 1) Following is the first version. Here the it give wrong value for >>>> count >>>> >>>> from StratosHealth as h unidirectional join liveInstances as ls on >>>> h.domain==ls.domain select count(ls.ip) as instanceCount, h.connections as >>>> connections group by ls.ip insert into AutoscaleData for current-events >>>> >>>> 2) Here the count is OK, but it send an event out once per each value >>>> in event table. >>>> >>>> from StratosHealth#window.length(0) as h join liveInstances as ls on >>>> h.domain==ls.domain select count(ls.ip) as instanceCount, h.connections as >>>> connections group by ls.ip insert into AutoscaleData for current-events >>>> >>>> Pamod, Manoj FYI, this is the issue. >>>> >>>> --Srinath >>>> >>>> >>>> >>>> On Mon, Sep 30, 2013 at 1:24 PM, Srinath Perera <[email protected]>wrote: >>>> >>>>> Yes, I thought it should only count things generated by Join >>>>> >>>>> --Srinath >>>>> >>>>> >>>>> On Mon, Sep 30, 2013 at 1:02 PM, Rajeev Sampath <[email protected]>wrote: >>>>> >>>>>> On Mon, Sep 30, 2013 at 12:17 PM, Srinath Perera <[email protected]>wrote: >>>>>> >>>>>>> Hi Suho, >>>>>>> >>>>>>> Could you explain why? Is it a bug or did we screw up something in >>>>>>> the language? >>>>>>> >>>>>>> I could not figure why we need "*all-events*" and I guess same for >>>>>>> most users. >>>>>>> >>>>>>> >>>>>> Hi Suho, Srinath, >>>>>> >>>>>> As far as I understood: >>>>>> >>>>>> The right side window contains 2 events when the join happens. Since >>>>>> this is a unidirectional join, events arrive thru the second stream >>>>>> doesn't >>>>>> trigger the join. >>>>>> >>>>>> When the first trigger event arrives: >>>>>> it (i.e. the 'current-event') creates 2 events in the join. First >>>>>> event's count is 1. >>>>>> Second one's count is 2 (the output aggregator accumulates). >>>>>> There are no 'expired-events' at the moment since by this time left >>>>>> window is empty. >>>>>> >>>>>> Then the second trigger event arrives: >>>>>> The 'current-event' creates 2 events in the join. First one's count >>>>>> is 3. (accumulated with previous one). Second one's count is 4. (again >>>>>> accumulated with previous) >>>>>> >>>>>> Then this also emits an expired-event (coz left window size is 1). >>>>>> Due to this expired-event another 2 events created in the join >>>>>> operation. >>>>>> First one has count 3 (subtracted from accumulated values due to >>>>>> event type) >>>>>> The second one results in a count of 2 (subtracted again) >>>>>> >>>>>> As I noticed, the count output aggregator keeps incrementing and >>>>>> decrementing its value depending on event type. >>>>>> Pls correct if I'm wrong. >>>>>> >>>>>> I was initially thinking that count() will just count the number of >>>>>> rows the join operation will create. >>>>>> Is this the expected behaviour? >>>>>> >>>>>> >>>>>> Thanks >>>>>> Rajeev >>>>>> >>>>>> --Srinath >>>>>>> >>>>>>> >>>>>>> On Mon, Sep 30, 2013 at 11:51 AM, Sriskandarajah Suhothayan < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> >>>>>>>> make the output to be all-events/ expired-events in the query as >>>>>>>> below, then you will be able to get the expected value. >>>>>>>> >>>>>>>> siddhiManager1.addQuery("from StratosHealth#window.length(1) as t >>>>>>>> unidirectional join InstanceHealth#window.length(2) as h " >>>>>>>> +" on h.domain == t.domain " >>>>>>>> +" select count(h.ip) as instanceCount, ip, >>>>>>>> connections insert into outStream for *all-events*"); >>>>>>>> >>>>>>>> Regards >>>>>>>> Suho >>>>>>>> >>>>>>>> >>>>>>>> On Fri, Sep 27, 2013 at 6:16 AM, Sriskandarajah Suhothayan < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> Srinath can you send the query and the input please I'll go through >>>>>>>>> >>>>>>>>> Suho >>>>>>>>> On Sep 27, 2013 3:45 AM, "Srinath Perera" <[email protected]> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Thanks Rajeev, pls note this is bit urgent. >>>>>>>>>> >>>>>>>>>> --Srinath >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Fri, Sep 27, 2013 at 11:05 AM, Rajeev Sampath < >>>>>>>>>> [email protected]> wrote: >>>>>>>>>> >>>>>>>>>>> Hi Suho, >>>>>>>>>>> >>>>>>>>>>> Re attaching Srinath's test case for your reference. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks >>>>>>>>>>> Rajeev >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Thu, Sep 26, 2013 at 9:03 PM, Sriskandarajah Suhothayan < >>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>> >>>>>>>>>>>> please write a test case in siddhi and give a patch >>>>>>>>>>>> so I/We can run and see and verify >>>>>>>>>>>> >>>>>>>>>>>> Suho >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On Thu, Sep 26, 2013 at 12:45 AM, Rajeev Sampath < >>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Copying dev list. >>>>>>>>>>>>> >>>>>>>>>>>>> ---------- Forwarded message ---------- >>>>>>>>>>>>> From: Rajeev Sampath <[email protected]> >>>>>>>>>>>>> Date: Thu, Sep 26, 2013 at 1:12 PM >>>>>>>>>>>>> Subject: Re: CEP sample >>>>>>>>>>>>> To: Srinath Perera <[email protected]> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> It seems all aggregation functions over joins (including >>>>>>>>>>>>> window joins) are not functioning properly. They keep accumulating >>>>>>>>>>>>> everything over time. I'll look into this. >>>>>>>>>>>>> >>>>>>>>>>>>> For array index out of bounds, the delete query can be fixed >>>>>>>>>>>>> as follows: >>>>>>>>>>>>> >>>>>>>>>>>>> from InstanceHealth[action=='down'] >>>>>>>>>>>>> delete liveInstances >>>>>>>>>>>>> on ip==liveInstances.ip >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On Thu, Sep 26, 2013 at 6:46 AM, Srinath Perera < >>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Hi Rajeev, >>>>>>>>>>>>>> >>>>>>>>>>>>>> If I remove the following line, it still shows as 2 even with >>>>>>>>>>>>>> group by. >>>>>>>>>>>>>> >>>>>>>>>>>>>> stream1Handle1.send(new Object[]{"qsb", >>>>>>>>>>>>>> System.currentTimeMillis(), "up", "192.121.0.2"}); >>>>>>>>>>>>>> >>>>>>>>>>>>>> Also do you find why array index outof bound happend when I >>>>>>>>>>>>>> uncomment >>>>>>>>>>>>>> >>>>>>>>>>>>>> //stream1Handle1.send(new Object[]{"qsb", >>>>>>>>>>>>>> System.currentTimeMillis(), "down", "192.121.0.1"}); >>>>>>>>>>>>>> >>>>>>>>>>>>>> --Srinath >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Thu, Sep 26, 2013 at 12:02 AM, Rajeev Sampath < >>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Seems this is because group by is not used in the query. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> from StratosHealth as h unidirectional join liveInstances >>>>>>>>>>>>>>> as ls >>>>>>>>>>>>>>> on h.domain==ls.domain >>>>>>>>>>>>>>> select count(ls.ip) as instanceCount, h.connections as >>>>>>>>>>>>>>> connections >>>>>>>>>>>>>>> group by ls.ip >>>>>>>>>>>>>>> insert into AutoscaleData >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Again, this can still result in duplicate events since >>>>>>>>>>>>>>> siddhi will emit the events in the resulting joined table >>>>>>>>>>>>>>> instead of one >>>>>>>>>>>>>>> event. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Wed, Sep 25, 2013 at 4:02 PM, Srinath Perera < >>>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>> ============================ >>>>>>>>>>>>>>>> Srinath Perera, Ph.D. >>>>>>>>>>>>>>>> Director, Research, WSO2 Inc. >>>>>>>>>>>>>>>> Visiting Faculty, University of Moratuwa >>>>>>>>>>>>>>>> Member, Apache Software Foundation >>>>>>>>>>>>>>>> Research Scientist, Lanka Software Foundation >>>>>>>>>>>>>>>> Blog: http://srinathsview.blogspot.com/ >>>>>>>>>>>>>>>> Photos: http://www.flickr.com/photos/hemapani/ >>>>>>>>>>>>>>>> Phone: 0772360902 >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>> Rajeev Sampath >>>>>>>>>>>>>>> Senior Software Engineer >>>>>>>>>>>>>>> WSO2, Inc.; http://www.wso2.com. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Mobile:* +94716265766 >>>>>>>>>>>>>>> * >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> -- >>>>>>>>>>>>>> ============================ >>>>>>>>>>>>>> Srinath Perera, Ph.D. >>>>>>>>>>>>>> Director, Research, WSO2 Inc. >>>>>>>>>>>>>> Visiting Faculty, University of Moratuwa >>>>>>>>>>>>>> Member, Apache Software Foundation >>>>>>>>>>>>>> Research Scientist, Lanka Software Foundation >>>>>>>>>>>>>> Blog: http://srinathsview.blogspot.com/ >>>>>>>>>>>>>> Photos: http://www.flickr.com/photos/hemapani/ >>>>>>>>>>>>>> Phone: 0772360902 >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> -- >>>>>>>>>>>>> Rajeev Sampath >>>>>>>>>>>>> Senior Software Engineer >>>>>>>>>>>>> WSO2, Inc.; http://www.wso2.com. >>>>>>>>>>>>> >>>>>>>>>>>>> Mobile:* +94716265766 >>>>>>>>>>>>> * >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> -- >>>>>>>>>>>>> Rajeev Sampath >>>>>>>>>>>>> Senior Software Engineer >>>>>>>>>>>>> WSO2, Inc.; http://www.wso2.com. >>>>>>>>>>>>> >>>>>>>>>>>>> Mobile:* +94716265766 >>>>>>>>>>>>> * >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> *S. Suhothayan >>>>>>>>>>>> * >>>>>>>>>>>> Associate Technical Lead, >>>>>>>>>>>> *WSO2 Inc. *http://wso2.com * >>>>>>>>>>>> <http://wso2.com/>* >>>>>>>>>>>> lean . enterprise . middleware >>>>>>>>>>>> >>>>>>>>>>>> *cell: (+94) 779 756 757 | blog: http://suhothayan.blogspot.com/ >>>>>>>>>>>> >>>>>>>>>>>> twitter: http://twitter.com/suhothayan | linked-in: >>>>>>>>>>>> http://lk.linkedin.com/in/suhothayan* >>>>>>>>>>>> * >>>>>>>>>>>> * >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> Rajeev Sampath >>>>>>>>>>> Senior Software Engineer >>>>>>>>>>> WSO2, Inc.; http://www.wso2.com. >>>>>>>>>>> >>>>>>>>>>> Mobile:* +94716265766 >>>>>>>>>>> * >>>>>>>>>>> >>>>>>>>>>> _______________________________________________ >>>>>>>>>>> Dev mailing list >>>>>>>>>>> [email protected] >>>>>>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> ============================ >>>>>>>>>> Srinath Perera, Ph.D. >>>>>>>>>> Director, Research, WSO2 Inc. >>>>>>>>>> Visiting Faculty, University of Moratuwa >>>>>>>>>> Member, Apache Software Foundation >>>>>>>>>> Research Scientist, Lanka Software Foundation >>>>>>>>>> Blog: http://srinathsview.blogspot.com/ >>>>>>>>>> Photos: http://www.flickr.com/photos/hemapani/ >>>>>>>>>> Phone: 0772360902 >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> *S. Suhothayan >>>>>>>> * >>>>>>>> Associate Technical Lead, >>>>>>>> *WSO2 Inc. *http://wso2.com * >>>>>>>> <http://wso2.com/>* >>>>>>>> lean . enterprise . middleware >>>>>>>> >>>>>>>> *cell: (+94) 779 756 757 | blog: http://suhothayan.blogspot.com/ >>>>>>>> twitter: http://twitter.com/suhothayan | linked-in: >>>>>>>> http://lk.linkedin.com/in/suhothayan* >>>>>>>> * >>>>>>>> * >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> ============================ >>>>>>> Srinath Perera, Ph.D. >>>>>>> Director, Research, WSO2 Inc. >>>>>>> Visiting Faculty, University of Moratuwa >>>>>>> Member, Apache Software Foundation >>>>>>> Research Scientist, Lanka Software Foundation >>>>>>> Blog: http://srinathsview.blogspot.com/ >>>>>>> Photos: http://www.flickr.com/photos/hemapani/ >>>>>>> Phone: 0772360902 >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Rajeev Sampath >>>>>> Senior Software Engineer >>>>>> WSO2, Inc.; http://www.wso2.com. >>>>>> >>>>>> Mobile:* +94716265766 >>>>>> * >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> ============================ >>>>> Srinath Perera, Ph.D. >>>>> Director, Research, WSO2 Inc. >>>>> Visiting Faculty, University of Moratuwa >>>>> Member, Apache Software Foundation >>>>> Research Scientist, Lanka Software Foundation >>>>> Blog: http://srinathsview.blogspot.com/ >>>>> Photos: http://www.flickr.com/photos/hemapani/ >>>>> Phone: 0772360902 >>>>> >>>> >>>> >>>> >>>> -- >>>> ============================ >>>> Srinath Perera, Ph.D. >>>> Director, Research, WSO2 Inc. >>>> Visiting Faculty, University of Moratuwa >>>> Member, Apache Software Foundation >>>> Research Scientist, Lanka Software Foundation >>>> Blog: http://srinathsview.blogspot.com/ >>>> Photos: http://www.flickr.com/photos/hemapani/ >>>> Phone: 0772360902 >>>> >>> >>> >>> >>> -- >>> ============================ >>> Srinath Perera, Ph.D. >>> Director, Research, WSO2 Inc. >>> Visiting Faculty, University of Moratuwa >>> Member, Apache Software Foundation >>> Research Scientist, Lanka Software Foundation >>> Blog: http://srinathsview.blogspot.com/ >>> Photos: http://www.flickr.com/photos/hemapani/ >>> Phone: 0772360902 >>> >> > > > -- > ============================ > Srinath Perera, Ph.D. > Director, Research, WSO2 Inc. > Visiting Faculty, University of Moratuwa > Member, Apache Software Foundation > Research Scientist, Lanka Software Foundation > Blog: http://srinathsview.blogspot.com/ > Photos: http://www.flickr.com/photos/hemapani/ > Phone: 0772360902 > -- *S. Suhothayan * Associate Technical Lead, *WSO2 Inc. *http://wso2.com * <http://wso2.com/>* lean . enterprise . middleware *cell: (+94) 779 756 757 | blog: http://suhothayan.blogspot.com/ twitter: http://twitter.com/suhothayan | linked-in: http://lk.linkedin.com/in/suhothayan* * *
Join_Fix.patch
Description: Binary data
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
