Hi, > The main reason I write this mail is, because I have to access the last N elements of each stream for each new arriving element
You can not go back in a stream, so you have to write your own bolt that stores the last windows. That should be pretty straight forward. On Mon, Nov 10, 2014 at 12:26 PM, Johannes Hugo Kitschke < [email protected]> wrote: > Hi there, > > I want to solve a task and wonder if Storm is suitable for this. My task: > > - input (many) numerical datastreams (e.g. stock market data, seismic > data, ...) > - define (many) queries of length N (may be different for each query) > - compute distance (e.g. using dynamic time warping) between each > query and the last N datapoints of each datastream > - report matches if distance < eps > > This is a rough outline. I am completely new to Storm and I wonder, if > Storm is a good candidate to solve this problem (if not, alternatives?). > The main reason I write this mail is, because I have to access the last N > elements of each stream for each new arriving element. But every example I > found does only do some computation on one element at a time. > > While writing this, I came across the 'RollingCountBolt'. Does this > implement the functionality I want? Does this mean each Bolt doing the > distance computation for the last N points has to store these points? > > I would be interested in your thoughts, hints and ideas. > > Thanks! > Johannes >
