+1 .. excellent job getting this off the ground! I'd love to see the numbers in a real distributed set up :).
On Wed, Nov 27, 2013 at 1:47 PM, Srinath Perera <[email protected]> wrote: > Hi All, > > I have written a Siddhi bolt that you can use to run Siddhi using Storm in > a distributed setup. > > You can create a SiddhiBolt(s) given any Siddhi query like following. > > SiddhiBolt siddhiBolt = new SiddhiBolt( > new String[]{ "define stream PlayStream1 ( sid string, ts long, x > double, y double, z double, a double, v double);"}, > new String[]{ "from PlayStream1#window.timeBatch(1sec) select > sid, avg(v) as avgV insert into AvgRunPlay;" }, > new String[]{"AvgRunPlay"}); > > Then those bolts can be used within Storm topology like any other bolt. > However, the name of components and streams used in CEP queries should > match. > > TopologyBuilder builder = new TopologyBuilder(); > builder.setSpout("PlayStream1", new FootballDataSpout(), 1); > builder.setBolt("AvgRunPlay", siddhiBolt1, > 1).shuffleGrouping("PlayStream1"); > > builder.setBolt("FastRunPlay", siddhiBolt2,1).shuffleGrouping("AvgRunPlay"); > builder.setBolt("LeafEacho", new EchoBolt(), > 1).shuffleGrouping("FastRunPlay"); > > I have done a quick performance test and got about 140K TPS in local > cluster. We need to test using distributed setup. Lasantha will integrate > this with CEP code base. > > Some potential TODO are > 1) Write two new bolts for Siddhi that support reliable processing and > transaction processing using Storm constructs. (for cases where we need > high reliability while processing) > 2) Integrate this with out data agent so we can send events into Storm > setup as well. > 3) Extend the Siddhi language to support distributed processing, so above > topology can be written is Siddhi language itself. > > If performance confirmed to be in the same range, given stability of > Storm, I think we can go with Storm for planned Siddhi distributed > processing. > > Thanks > Srinath > > Code for the bolt can be found in > https://svn.wso2.org/repos/wso2/people/srinath/projects/siddhiStormIntegration/src/org/wso2/siddhi/storm/SiddhiBolt.java > . > > Code can be found from > https://svn.wso2.org/repos/wso2/people/srinath/projects/siddhiStormIntegration > > -- > ============================ > Srinath Perera, Ph.D. > Director, Research, WSO2 Inc. > Visiting Faculty, University of Moratuwa > Member, Apache Software Foundation > Research Scientist, Lanka Software Foundation > Blog: http://srinathsview.blogspot.com/ > Photos: http://www.flickr.com/photos/hemapani/ > Phone: 0772360902 > > _______________________________________________ > Architecture mailing list > [email protected] > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > > -- Sanjiva Weerawarana, Ph.D. Founder, Chairman & CEO; WSO2, Inc.; http://wso2.com/ email: [email protected]; office: +1 650 745 4499 x5700; cell: +94 77 787 6880 | +1 650 265 8311 blog: http://sanjiva.weerawarana.org/ Lean . Enterprise . Middleware
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
