I couldn't spot an issue in JIRA related to calling activate/deactivate, but happy to hear it works for you on 1.2.1.
Regarding HDInsight, I would contact Microsoft and ask them to provide a newer version of Storm. 1.1.0 is slightly over than a year old. Regarding your design, it sounds okay to me. Is the config cache present in both the bolts and the spout? Is it important that the caches all contain the same config, or can you update the bolts individually? e.g. for topology spout -> boltA -> boltB, is it okay if boltB updates its cache first, or do you need it to go update boltA, then update boltB? If the cache is only in the bolts, and you don't need the updates to flow from the spout, I would consider looking at tick tuples ( http://www.michael-noll.com/blog/2013/01/18/implementing-real-time-trending-topics-in-storm/#excursus-tick-tuples-in-storm-08) as an option to periodically refresh the cache. On the other hand, if you need the cache refreshes to be triggered by the spout, I would make the spout emit the update tuple periodically, plus on the first call to nextTuple after the spout is activated. If you emit the tuple every once in a while, you don't need to manually refresh the caches by cycling topology activation. You probably want to use acking for the cache refresh tuple, so you can ensure that all bolts manage to process it. 2018-05-10 7:31 GMT+02:00 Satyam Singh <[email protected]>: > Hi Stig, > > Thanks for you effort! > I tested my code on 1.2.1 version and it is working fine. deactivate and > activate are getting called on manual trigger. > This looks like bug in 1.1.0 version! > > Let me explain my requirement: > I am using Storm cluster on HDInsight (Azure) and unfortunately they have > not upgraded to 1.2.1 yet! > That means i can't use activate and deactivate on Azure using HDInsight. I > have to create my own cluster using VMs. > > Let me explain why i want this deactivate. > I have to update some config data cache initialized during topology start > in bolts. As per my understanding this can be done only by deactivate spout > and send one update tuple from spout to bolts so that bolts can also update > cache (read updated config data from db). Then activate spout and it will > resume topology with updated cache. > > Let me know your thoughts for my above approach of cache update. > > > Regards, > Satyam > > On Wed, May 9, 2018 at 3:11 AM, Stig Rohde Døssing <[email protected]> > wrote: > >> I checked that this works in a single-node install using Storm 1.2.1. I >> used this topology https://github.com/apache/stor >> m/blob/v1.2.1/examples/storm-starter/src/jvm/org/apache/ >> storm/starter/WordCountTopology.java and this spout >> https://github.com/apache/storm/blob/v1.2.1/examples/storm- >> starter/src/jvm/org/apache/storm/starter/spout/RandomSentenceSpout.java, >> and added a couple of error level logs to the deactivate/activate methods >> of the spout. Running the topology prints the log from activate. >> Deactivating the topology via the CLI correctly deactivates the topology >> (as shown in Storm UI) and the log from the deactivate method is printed to >> the worker log. Similarly, activating the topology resumes the topology and >> prints the activate log to the worker log. >> >> I would try the following: >> Verify that the deactivate method is definitely not getting called by >> crashing the spout if it is (just throw an exception). That should make >> completely sure that the log isn't just disappearing somehow, since a >> worker crash is very visible. >> Try upgrading to the latest Storm version (1.2.1) to see if that solves >> it. If you're on Windows, also try using the Powershell script (storm.ps1) >> to interact with Storm instead of the storm.cmd script. >> If not, see if you can reproduce the issue with the example topology I >> linked above. This would let us rule out an issue with the topology code. >> >> 2018-05-08 19:11 GMT+02:00 Satyam Singh <[email protected]>: >> >>> Its Regular >>> >>> Sent from satyam's iPhone >>> >>> On 08-May-2018, at 10:26 PM, Stig Rohde Døssing <[email protected]> wrote: >>> >>> Thanks. I'll try to reproduce it. What kind of spout/topology is it >>> (e.g. "regular", Trident, multilang)? >>> >>> 2018-05-08 18:51 GMT+02:00 Satyam Singh <[email protected]>: >>> >>>> Hi Stig, >>>> >>>> I am using 1.1.0 and i have put some log prints in deactivate and >>>> activate methods. >>>> During start of topology log prints of activate method are coming but >>>> when i manually trigger deactivate and activate via command line then >>>> neither deactivate nor activate prints are coming. >>>> >>>> On Tue, May 8, 2018 at 4:08 PM, Stig Rohde Døssing <[email protected]> >>>> wrote: >>>> >>>>> Which version are you using? Also how are you verifying that the >>>>> deactivate method isn't getting called? >>>>> >>>>> 2018-05-08 11:13 GMT+02:00 Satyam Singh <[email protected]>: >>>>> >>>>>> Hi All, >>>>>> >>>>>> >>>>>> Issue facing: >>>>>> storm deactivate <topology-name> >>>>>> running fine and topology mentioned is deactivated but i have written >>>>>> some logic in deactivate method of spout and it is not getting called. >>>>>> >>>>>> If anyone have face same issue. Suggest me what to do. I have tried >>>>>> multiple time but no luck! >>>>>> >>>>>> >>>>>> Thanks >>>>>> Satyam >>>>>> >>>>>> >>>>> >>>> >>> >> >
