That would be great... in our case, we have replaced one stream with several time window queries with several streams and queries that limit the output rate.
Previous design was: from STREAM_ALL time_window(1m) from STREAM_ALL time_window(1h) from STREAM_ALL time_window(1d) Current design is: from STREAM_ALL time_window(30s) output snapshot every 30s, insert into STREAM_MINUTE from STREAM_MINUTE output last every 60 events, insert into STREAM_HOUR from STREAM_HOUR output last every 24 events, insert into STREAM_DAY This way we can aggregate data per time range and reduce the number of events. Do you think that this is an appropiate design for this scenario? Regards 2014-04-23 10:49 GMT+02:00 Sriskandarajah Suhothayan <[email protected]>: > Thanks for the update. > > We also have ideas to implement windows that will use file system to off > load some data when the data load is high (e.g one day). With this you will > be able to process huge data effectively. > > Regards > Suho > > > On Wed, Apr 23, 2014 at 1:58 PM, David Morales <[email protected]>wrote: > >> Thanks for your replies, we have done more tests and we can confirm that >> this is not an issue related to Siddhi. >> >> Our problem was related to the use of external time windows and the >> dataset we are testing with. Anyway, we have redesign our streams and >> queries to avoid the use of very large windows (one day, for example). >> >> >> Regards. >> >> >> >> >> 2014-04-22 5:03 GMT+02:00 Srinath Perera <[email protected]>: >> >> If it is possible to create a testcase reproducing the issue, that will >>> be much useful. >>> >>> >>> On Tue, Apr 22, 2014 at 8:28 AM, Sriskandarajah Suhothayan < >>> [email protected]> wrote: >>> >>>> Please create a ticket in the CEP Jira[1] and add your findings >>>> We'll investigate on this. >>>> >>>> Thanks >>>> Suho >>>> >>>> [1]https://wso2.org/jira/browse/CEP >>>> >>>> >>>> On Mon, Apr 21, 2014 at 11:04 PM, David Morales >>>> <[email protected]>wrote: >>>> >>>>> Hi, >>>>> >>>>> We have suffering memory leaks when using queries with time batch >>>>> windows. >>>>> >>>>> As fas as we have seen, several LinkedBlockingQueue objects are being >>>>> created. >>>>> >>>>> Our code is using siddhi as external library and we are not using >>>>> hazelcast, do you know any related issue? >>>>> >>>>> >>>>> Regards. >>>>> >>>>> _______________________________________________ >>>>> Architecture mailing list >>>>> [email protected] >>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>> >>>>> >>>> >>>> >>>> -- >>>> >>>> *S. Suhothayan * >>>> Associate Technical Lead, >>>> *WSO2 Inc. *http://wso2.com >>>> * <http://wso2.com/>* >>>> lean . enterprise . middleware >>>> >>>> >>>> *cell: (+94) 779 756 757 <%28%2B94%29%20779%20756%20757> | blog: >>>> http://suhothayan.blogspot.com/ <http://suhothayan.blogspot.com/> twitter: >>>> http://twitter.com/suhothayan <http://twitter.com/suhothayan> | linked-in: >>>> http://lk.linkedin.com/in/suhothayan >>>> <http://lk.linkedin.com/in/suhothayan>* >>>> >>>> >>>> _______________________________________________ >>>> Architecture mailing list >>>> [email protected] >>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>> >>>> >>> >>> >>> -- >>> ============================ >>> Srinath Perera, Ph.D. >>> http://people.apache.org/~hemapani/ >>> http://srinathsview.blogspot.com/ >>> >>> _______________________________________________ >>> Architecture mailing list >>> [email protected] >>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>> >>> >> >> _______________________________________________ >> Architecture mailing list >> [email protected] >> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >> >> > > > -- > > *S. Suhothayan* > Associate Technical Lead, > *WSO2 Inc. *http://wso2.com > * <http://wso2.com/>* > lean . enterprise . middleware > > > *cell: (+94) 779 756 757 | blog: http://suhothayan.blogspot.com/ > <http://suhothayan.blogspot.com/>twitter: http://twitter.com/suhothayan > <http://twitter.com/suhothayan> | linked-in: > http://lk.linkedin.com/in/suhothayan <http://lk.linkedin.com/in/suhothayan>* > > > _______________________________________________ > Architecture mailing list > [email protected] > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > >
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
