Hi, We (myself and Srinath) had a chat about the complexity of implementing a solution for the issue of unique identifiers for Siddhi queries. Since this task has multiple sub problems to be solved (for example to what extent the Siddhi applications are different) it has large scope. Hence we will not continue investigating this issue immediately but will table it as a future project.
-- Thanks, Miyuru Dayarathna Senior Technical Lead Mobile: +94713527783 Blog: http://miyurublog.blogspot.com On Tue, Mar 6, 2018 at 10:05 AM, Miyuru Dayarathna <miyu...@wso2.com> wrote: > Hi, > > Snapshotable [1] is the parent interface in Siddhi which should be > implemented by every processing element of a Siddhi event processing chain > to enable their state to be persisted and restored. Snapshotable > interface's getElementId() method returns unique element identifiers > (i.e., element ID) for each of the elements in the Siddhi query. > > @app:name('Test') > define stream StockStream ( symbol string, price float, volume int ); > @info(name = 'query1') > from StockStream[price>10]#window.length(10) > select symbol, price, sum(volume) as totalVol > insert into OutStream > > For a Siddhi query such as the one shown above, the length window may get > assigned an element ID such as "AbstractStreamProcessor-Test-1". The > format of this element ID is <Element_Type>-<Siddhi_App_ > Name>-<unique_integer>. > > These unique element identifiers are unique only for that particular > Siddhi query. Such characteristics is completely fine as long as the Siddhi > application does not change. However, if new elements get added to the > Siddhi query, there is a rare possibility of loading invalid state into the > stream application. An example of such scenario has been reported in [2]. > Similar issues would have to be faced in scenarios such as upgrades and > rollbacks, cross datacenter failover, state archiving, application > migration, etc. One solution for this issue is to keep track of the changes > made to the Siddhi application while maintaining a permanent unique ID > across the application's life cycle. If for example a new segment Y was > added to a Siddhi application X which creates a new application XY, we > could load the state of the already checkpointed segment (X) to the new > Siddhi application (XY) and then assign new IDs for the elements in the > newly added segment (Y). If we delete a segment Z from XY, then we will be > loading only elements pertaining to the remaining portion. The checkpoints > corresponding to the segment Z will be eventually cleaned by the state > management mechanism. > > [1] https://github.com/wso2/siddhi/blob/master/modules/ > siddhi-core/src/main/java/org/wso2/siddhi/core/util/ > snapshot/Snapshotable.java > > [2] https://github.com/wso2/siddhi/issues/776 > > -- > Thanks, > Miyuru Dayarathna > Senior Technical Lead > Mobile: +94713527783 <+94%2071%20352%207783> > Blog: http://miyurublog.blogspot.com > -- Thanks, Miyuru Dayarathna Senior Technical Lead Mobile: +94713527783 Blog: http://miyurublog.blogspot.com
_______________________________________________ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
