Also there's no config file that can do something similar right? It has to be done in the code?
> On Aug 15, 2017, at 14:31, Mahak Goel <[email protected]> wrote: > > Thanks stig, that worked for me! > > Another question, how does storm internally handle this time out? Is there > some source code you can point me to? > > Sent from my iPhone > >> On Aug 15, 2017, at 12:15, Stig Rohde Døssing <[email protected]> wrote: >> >> I think you need to give the FQCN for SleepSpoutWaitStrategy instead of an >> instance, since the config must be serializable to JSON, a little surprised >> you don't get an error when you submit that topology. If you're using the >> default wait strategy, you can just leave out the >> TOPOLOGY_SPOUT_WAIT_STRATEGY part. >> >> Here's what works for me (based on the word count topology in storm-starter): >> >> builder.setSpout("spout", new RandomSentenceSpout(), 5) >> .addConfiguration(Config.TOPOLOGY_SPOUT_WAIT_STRATEGY, new >> TestWait().getClass().getName()) >> .addConfiguration(Config.TOPOLOGY_SLEEP_SPOUT_WAIT_STRATEGY_TIME_MS, >> 60_000); >> >> where TestWait is just an inner class like this (purely so I can print the >> configuration, normally I'd just use the built in wait strategy) >> >> public static final class TestWait extends SleepSpoutWaitStrategy { >> >> @Override >> public void prepare(Map<String, Object> conf) { >> super.prepare(conf); >> LogManager.getLogger(getClass()).error("The sleep backoff is >> {}", conf.get(Config.TOPOLOGY_SLEEP_SPOUT_WAIT_STRATEGY_TIME_MS)); >> } >> >> } >> >> When I run the topology I get the following in the log: >> 2017-08-15 18:11:56.596 o.a.s.s.WordCountTopology$TestWait main [ERROR] The >> sleep backoff is 60000 >> >> 2017-08-15 18:00 GMT+02:00 Mahak Goel <[email protected]>: >>> In the last line I use addConfigurations >>> >>> Sent from my iPhone >>> >>>> On Aug 15, 2017, at 11:59, Mahak Goel <[email protected]> wrote: >>>> >>>> Hmm okay, that's what I'm trying to do but maybe I'm doing it wrong. >>>> >>>> >>>> Config config = new Config(); >>>> SleepSpoutWaitStrategy strategy = new SleepSpoutWaitStrategy(); >>>> config.put(org.apache.storm.Config.TOPOLOGY_SPOUT_WAIT_STRATEGY, strategy); >>>> config.put(org.apache.storm.Config.TOPOLOGY_SLEEP_SPOUT_WAIT_STRATEGY_TIME_MS, >>>> 10); >>>> builder.setSpout(...).addConfiguration(config); >>>> >>>> >>>> >>>> Sent from my iPhone >>>> On Aug 15, 2017, at 11:51, Stig Rohde Døssing <[email protected]> wrote: >>>> >>>>> I think I might have misread the code. It looks like the method I linked >>>>> does the opposite of what I thought, and removes only the configuration >>>>> that is not listed in the link. I would expect using >>>>> SpoutDeclarer.addConfiguration to work then. >>>>> >>>>> 2017-08-15 17:36 GMT+02:00 Mahak Goel <[email protected]>: >>>>>> Text from post. >>>>>> >>>>>> 2. Spout wait strategies: There's two situations in which a spout needs >>>>>> to wait. The first is when the max spout pending limit is reached. The >>>>>> second is when nothing is emitted from nextTuple. Previously, Storm >>>>>> would just have that spout sit in a busy loop in those cases. What Storm >>>>>> does in those situations is now pluggable, and the default is now for >>>>>> the spout to sleep for 1 ms. This will cause the spout to use >>>>>> dramatically less CPU when it hits those cases, and it also obviates the >>>>>> need for spouts to do any sleeping in their implementation to be >>>>>> "polite". The wait strategy can be configured with >>>>>> TOPOLOGY_SPOUT_WAIT_STRATEGY and can be configured on a spout by spout >>>>>> basis. The interface to implement for a wait strategy is >>>>>> backtype.storm.spout.ISpoutWaitStrategy >>>>>> >>>>>> >>>>>> >>>>>>> On Aug 15, 2017, at 11:34, Mahak Goel <[email protected]> wrote: >>>>>>> >>>>>>> I tried adding TOPOLOGY_SPOUT_WAIT_STRATEGY and >>>>>>> TOPOLOGY_SLEEP_SPOUT_WAIT_STRATEGY_TIME_MS in the spouts config but >>>>>>> that didn't seem to have an effect. >>>>>>> >>>>>>> >>>>>>>> On Aug 15, 2017, at 11:28, Mahak Goel <[email protected]> wrote: >>>>>>>> >>>>>>>> Hi Stig, >>>>>>>> >>>>>>>> Thank you. However it looks like from this post there is a way to do >>>>>>>> it on a per spout basis. >>>>>>>> https://groups.google.com/forum/m/#!search/Storm$200.8.1$20released/storm-user/hVbXtBdCkQo >>>>>>>> >>>>>>>> Do you or does anyone else know if this is still a possibility? If so, >>>>>>>> how do I do it? >>>>>>>> >>>>>>>> >>>>>>>>> On Aug 15, 2017, at 11:14, Stig Rohde Døssing <[email protected]> wrote: >>>>>>>>> >>>>>>>>> Hi Mahak, >>>>>>>>> >>>>>>>>> I haven't checked in any detail, but I suspect there isn't. I'd have >>>>>>>>> said you could set the configuration for the spout via the >>>>>>>>> SpoutDeclarer addConfiguration methods when declaring the spout, but >>>>>>>>> it looks like the wait strategy and backoff are both removed from the >>>>>>>>> component configuration, and only read from the topology level >>>>>>>>> configuration >>>>>>>>> https://github.com/apache/storm/blob/64e29f365c9b5d3e15b33f33ab64e200345333e4/storm-client/src/jvm/org/apache/storm/executor/Executor.java#L431. >>>>>>>>> >>>>>>>>> >>>>>>>>> 2017-08-15 16:45 GMT+02:00 Brian Taylor >>>>>>>>> <[email protected]>: >>>>>>>>>> Unsubscribe >>>>>>>>>> >>>>>>>>>> Sent from BlueMail >>>>>>>>>>> On Aug 15, 2017, at 10:34 AM, Mahak Goel <[email protected]> >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>> Hi, >>>>>>>>>>> >>>>>>>>>>> I know I can configure a sleep wait strategy in the defaults.yaml >>>>>>>>>>> and that will apply to all spouts in the topology. Is there a way >>>>>>>>>>> to do this on a spout by spout basis? That is, is there a way to >>>>>>>>>>> configure different times for different spouts? >>>>>>>>>>> >>>>>>>>>>> Thanks! >>>>>>>>> >>>>> >>
