Thanks Cody, patching the ScopesImpl class is indeed the root solution, I'll see if I can do that.
Meanwhile, is there a way for me to automatically kill/resubmit my topology when I encounter this error? I realized that exiting when an exception is caught is not enough, the workers are restarted but the topology remains in a bogus state. Thanks. 2014-05-22 23:32 GMT+02:00 Cody A. Ray <[email protected]>: > I think the problem is that the code bundled with your topology is > serialized by the nimbus and transferred to the supervisors, so all the > code you use needs to be serializable. Something in twitter4j must be a > using a non-serializable class (the ScopesImpl class). > > I don't have a suggestion other than patch/contribute twitter4j or look at > a different twitter library... But I think that's the root cause. > > Someone else may have a better suggestion. > > -Cody > On May 22, 2014 1:00 PM, "Alexis Sukrieh" <[email protected]> wrote: > >> Hi, >> >> I have built a topology with a spout that connects to Twitter's streaming >> API (with twitter4j), I have also 4 bolts that count words inside. >> >> This is mainly a proof of concept for me, in order to get up to speed >> with Storm. >> >> So far, it's working fine... until my Spout gets an exception from >> twitter4j: >> >> java.lang.RuntimeException: java.lang.RuntimeException: >> java.io.NotSerializableException: twitter4j.ScopesImpl >> >> When this happens, my topology is somehow broken: the Redis database that >> gets updated by the last bolt is not updated anymore. >> >> >> All I can do is killing the topology and submit it again. That's a bit >> annoying as I'd like my topology to never stops running. >> >> I'd like to understand how to catch this exception, and make sure my >> topology is protected against invalid tuples. >> >> >> In the Spout class, I really don't see how I could catch the exception. >> >> Any help is more than welcome. >> >> >> Thanks. >> >> >> -- >> Alexis Sukrieh >> >> -- Alexis Sukrieh http://twitter.com/sukria - http://blog.sukria.net "People get annoyed when you try to debug them." -- Larry Wall
