That would slow down all of your spouts by a lot. If you just want it for a single kafkaspout then you would want to set it only for that spout by calling `addConfiguration(Config.TOPOLOGY_SLEEP_SPOUT_WAIT_STRATEGY_TIME_MS, 3000)` on the SpoutDeclarer for that spout.
The issue is that the spout sleeps that amount of time when there is an empty emit or if max spout pending was hit, or if back pressure said that the topology should be throttled. Not emitting things is very common, even when you are processing a normal amount of data. So for normal spouts you are likely to see the spout pause for 3 seconds (your setting), then get a big burst of data to process and if all 3 seconds of data cannot fit into topology.max.spout.pending the spout will sleep again for 3 seconds and now you have more then 3 seconds of data to process, which it is likely to now be able to do. - Bobby On Friday, July 21, 2017, 7:17:16 AM CDT, Stig Rohde Døssing <[email protected]> wrote: Yes, that should work too. 2017-07-21 13:35 GMT+02:00 I PVP <[email protected]>: Would defining it for each topology with the following code be also a option or is there any disadvantage of doing it this way? --org.apache.storm.Config conf = new Config();….conf.put(Config.TOPOLOGY_ SLEEP_SPOUT_WAIT_STRATEGY_ TIME_MS, 3000);-- best,IPVP On July 21, 2017 at 4:21:52 AM, Stig Rohde Døssing ([email protected]) wrote: When a call to nextTuple on the spout doesn't emit any tuples, the spout executor will sleep for a bit. The duration is set herehttps://github.com/apache/stor m/blob/e38f936077ea9b3ba5cd568 b69335e0aac8369dd/conf/ defaults.yaml#L247, you could increase it if you want. 2017-07-21 3:18 GMT+02:00 I PVP <[email protected]>: I am experiencing very High CPU usage with storm-kafka spout even when idle for hours. I changed all my Kafka Spouts to the new org.apache.storm.kafka.spo ut.KafkaSpout but the issue continues. How to tune it ?Is there something like a Utils.sleep for KafkaSpout? Thanks IP VP
