Thank you Jungtaek! Yup. You are exactly right. I was too tired or something and I didn't realize that the error is complaining about the inner class. Thank you for taking your time to look into this.
Regards, Gergely 2015-12-14 23:41 GMT+09:00 임정택 <[email protected]>: > Hi Gergely, > > Thanks for using storm-redis, and report issue. > > The thing Storm was trying to serialize is BinaryJedis$SetFromList, not > BinaryJedis. > > > https://github.com/Fogetti/fogetti-phish-storm/blob/master/src/main/java/fogetti/phish/storm/relatedness/GoogleSemBolt.java#L74 > > Jedis.smembers() returns SetFromList, which wraps List to provide Set > interface. > (It is just one of the optimizations in Jedis.) > > Unfortunately SetFromList doesn't implement Serializable for now, so you > can encounter NotSerializableException when trying to emit the result of > smembers directly. > > Two workarounds are here, > > 1. Implement custom Kyro serializer for SetFromList and register to > topology. > http://storm.apache.org/documentation/Serialization.html > > 2. Copy the result into HashSet and emit instead. > > I'll try to craft the patch to Jedis which makes SetFromList to implement > Serializable. > Before applying and releasing, I'll also try to make storm-redis to > provide easy workaround. (Considering 1. for now.) > > Thanks! > Jungtaek Lim (HeartSaVioR) > > > > > > 2015-12-14 21:54 GMT+09:00 Gergely Nagy <[email protected]>: > >> Hello fellow storm troopers. >> >> I am trying to build a storm topology in Storm 0.10.0. I am also trying >> to use storm-redis to save some interim results during operation. >> >> Now my problem is that when I run my topology locally everything works >> fine, but when I run it remotely, storm sends me this nice stacktrace: >> https://gist.github.com/Fogetti/23304178a534fa1f8f63 >> >> I don't get it. What's going on? Why is storm serializing BinaryJedis? >> Shouldn't BinaryJedis be transient? And anyway, what is this cryptic thing >> called 'consumeBatchToCursor()' doing? >> >> The "offending" class can be seen below. It's just a simple bolt >> extending AbstractRedisBolt. >> >> https://github.com/Fogetti/fogetti-phish-storm/blob/master/src/main/java/fogetti/phish/storm/relatedness/GoogleSemBolt.java >> >> Does anyone know what might be the root cause for this? >> >> Regards, >> Gergely >> > > > > -- > Name : 임 정택 > Blog : http://www.heartsavior.net / http://dev.heartsavior.net > Twitter : http://twitter.com/heartsavior > LinkedIn : http://www.linkedin.com/in/heartsavior >
