Thanks Arun, exactly what I needed! I’m going to create a ticket and a pull request with the updated code tomorrow morning (UTC+01:00) in order to move the discussion forward.
— Balázs > On 2016. máj. 30., at 21:41, Arun Mahadevan <ar...@apache.org> wrote: > > Hi Balázs, > > The idea sounds good. > > Only whitelisted configs can be overridden at component level. The java > configs are converted to clojure variables by replacing _ (underscore) with - > (hyphen) and you need to add and entry like [1] in executor.clj to get it > working. > > - Arun > > [1] > https://github.com/apache/storm/blob/master/storm-core/src/clj/org/apache/storm/daemon/executor.clj#L114 > > > > > On 5/30/16, 11:26 PM, "Balázs Kossovics" <balazs.kossov...@s4m.io> wrote: > >> Hey, >> >> I'm trying to implement an alterative behaviour in case of late tuples >> for windowing. Currently late tuples are just logged (and acknowledged >> in the coming 1.0.2), but in my usecase it would be desirable to emit >> them onto a user defined stream. One could define a bolt with a stream >> for late tuples like this: >> >> new MyWindowedBolt() >> .withTimestampField("timestamp") >> * .withLateTupleStream("late_tuples") >> .withWindow( >> new BaseWindowedBolt.Duration(1, TimeUnit.MINUTES), >> new BaseWindowedBolt.Duration(1, TimeUnit.SECONDS) >> ) >> >> I made a quick patch >> (https://github.com/kosii/storm/commit/216c991da3c5b6c6cac1b25182b86507c3fb5e9e) >> >> to test the idea, where the indended behaviour would be something like this: >> 1, the withLateTupleStream builder method puts a new key into the >> windowConfiguration Map, >> 2, in WindowedBoltExecutor. declareOutputFields a new stream gets declared, >> 3, in WindowedBoltExecutor.prepare we store the stream's name in a >> private variable, >> 4, if this variable is not null, then in the execute method we emit each >> late tuple onto the new stream. >> >> The problem is in the 3rd step >> (https://github.com/kosii/storm/commit/216c991da3c5b6c6cac1b25182b86507c3fb5e9e#diff-32eff130ad25f4b7b6c069e8d42245acR170), >> >> where the stormConf map doesn't contain anymore my key, which was still >> present in the 2nd step. I'm out of ideas, so I'd really appreciate if >> someone could explain me what's happening here. >> >> I'm also interested in your ideas concerning the feature, and what would >> you like to eventually see in the upstream. >> >> Best regards, >> Balazs >> >> >
