On Wed, Apr 23, 2014 at 2:36 PM, David Morales <[email protected]> wrote:
> That would be great... in our case, we have replaced one stream with > several time window queries with several streams and queries that limit the > output rate. > > Previous design was: > > from STREAM_ALL time_window(1m) > from STREAM_ALL time_window(1h) > from STREAM_ALL time_window(1d) > > Current design is: > > from STREAM_ALL time_window(30s) output snapshot every 30s, insert into > STREAM_MINUTE > from STREAM_MINUTE output last every 60 events, insert into STREAM_HOUR > from STREAM_HOUR output last every 24 events, insert into STREAM_DAY > > > This way we can aggregate data per time range and reduce the number of > events. Do you think that this is an appropiate design for this scenario? > +1 this seems to be a great design for this scenario. Go ahead with this. Suho > > > Regards > > > > > > > > 2014-04-23 10:49 GMT+02:00 Sriskandarajah Suhothayan <[email protected]>: > > Thanks for the update. >> >> We also have ideas to implement windows that will use file system to off >> load some data when the data load is high (e.g one day). With this you will >> be able to process huge data effectively. >> >> Regards >> Suho >> >> >> On Wed, Apr 23, 2014 at 1:58 PM, David Morales <[email protected]>wrote: >> >>> Thanks for your replies, we have done more tests and we can confirm that >>> this is not an issue related to Siddhi. >>> >>> Our problem was related to the use of external time windows and the >>> dataset we are testing with. Anyway, we have redesign our streams and >>> queries to avoid the use of very large windows (one day, for example). >>> >>> >>> Regards. >>> >>> >>> >>> >>> 2014-04-22 5:03 GMT+02:00 Srinath Perera <[email protected]>: >>> >>> If it is possible to create a testcase reproducing the issue, that will >>>> be much useful. >>>> >>>> >>>> On Tue, Apr 22, 2014 at 8:28 AM, Sriskandarajah Suhothayan < >>>> [email protected]> wrote: >>>> >>>>> Please create a ticket in the CEP Jira[1] and add your findings >>>>> We'll investigate on this. >>>>> >>>>> Thanks >>>>> Suho >>>>> >>>>> [1]https://wso2.org/jira/browse/CEP >>>>> >>>>> >>>>> On Mon, Apr 21, 2014 at 11:04 PM, David Morales >>>>> <[email protected]>wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> We have suffering memory leaks when using queries with time batch >>>>>> windows. >>>>>> >>>>>> As fas as we have seen, several LinkedBlockingQueue objects are being >>>>>> created. >>>>>> >>>>>> Our code is using siddhi as external library and we are not using >>>>>> hazelcast, do you know any related issue? >>>>>> >>>>>> >>>>>> Regards. >>>>>> >>>>>> _______________________________________________ >>>>>> Architecture mailing list >>>>>> [email protected] >>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> >>>>> *S. Suhothayan * >>>>> Associate Technical Lead, >>>>> *WSO2 Inc. *http://wso2.com >>>>> * <http://wso2.com/>* >>>>> lean . enterprise . middleware >>>>> >>>>> >>>>> *cell: (+94) 779 756 757 <%28%2B94%29%20779%20756%20757> | blog: >>>>> http://suhothayan.blogspot.com/ <http://suhothayan.blogspot.com/> twitter: >>>>> http://twitter.com/suhothayan <http://twitter.com/suhothayan> | linked-in: >>>>> http://lk.linkedin.com/in/suhothayan >>>>> <http://lk.linkedin.com/in/suhothayan>* >>>>> >>>>> >>>>> _______________________________________________ >>>>> Architecture mailing list >>>>> [email protected] >>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>> >>>>> >>>> >>>> >>>> -- >>>> ============================ >>>> Srinath Perera, Ph.D. >>>> http://people.apache.org/~hemapani/ >>>> http://srinathsview.blogspot.com/ >>>> >>>> _______________________________________________ >>>> Architecture mailing list >>>> [email protected] >>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>> >>>> >>> >>> _______________________________________________ >>> Architecture mailing list >>> [email protected] >>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>> >>> >> >> >> -- >> >> *S. Suhothayan* >> Associate Technical Lead, >> *WSO2 Inc. *http://wso2.com >> * <http://wso2.com/>* >> lean . enterprise . middleware >> >> >> *cell: (+94) 779 756 757 <%28%2B94%29%20779%20756%20757> | blog: >> http://suhothayan.blogspot.com/ <http://suhothayan.blogspot.com/> twitter: >> http://twitter.com/suhothayan <http://twitter.com/suhothayan> | linked-in: >> http://lk.linkedin.com/in/suhothayan <http://lk.linkedin.com/in/suhothayan>* >> >> >> _______________________________________________ >> Architecture mailing list >> [email protected] >> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >> >> > > _______________________________________________ > Architecture mailing list > [email protected] > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > > -- *S. Suhothayan* Associate Technical Lead, *WSO2 Inc. *http://wso2.com * <http://wso2.com/>* lean . enterprise . middleware *cell: (+94) 779 756 757 | blog: http://suhothayan.blogspot.com/ <http://suhothayan.blogspot.com/>twitter: http://twitter.com/suhothayan <http://twitter.com/suhothayan> | linked-in: http://lk.linkedin.com/in/suhothayan <http://lk.linkedin.com/in/suhothayan>*
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
