I built the v0.9.2-incubating rc-3 locally and once verifying that it
worked for our topology, pushed it into our cluster. So far, so good.

One thing for the community to be aware of. If you try to upgrade an
existing v0.9.1-incubating or 0.8 cluster to v0.9.2-incubating, you may hit
exceptions upon nimbus/supervisor startup about stormcode.ser/stormconf.ser.

The issue is that the new cluster will try to re-submit the topologies that
were already running before the upgrade. These will fail because Storm's
Clojure version has been upgraded from 1.4 -> 1.5, thus the serialization
formats & IDs have changed. This would be true basically if any class
serial IDs change that happen to be in these .ser files (stormconf.ser &
stormcode.ser, as defined in Storm's internal config
<https://github.com/apache/incubator-storm/blob/master/storm-core/src/clj/backtype/storm/config.clj#L143-L153>
).

The solution is to clear out the storm data directories on your worker
nodes/nimbus nodes and restart the cluster.

I have some open source tooling that submits topologies to the nimbus using
StormSubmitter. This upgrade also made me realize that due to the use
of serialized
Java files
<https://github.com/apache/incubator-storm/blob/master/storm-core/src/jvm/backtype/storm/utils/Utils.java#L73-L97>,
it is very important the StormSubmitter class used for submitting and the
running Storm cluster be precisely the same version / classpath. I describe
this more in the GH issue here:

https://github.com/Parsely/streamparse/issues/27

I wonder if maybe it's worth it to consider using a less finicky
serialization format within Storm itself. Would that change be welcome as a
pull request?

It would make it easier to script Storm clusters without consideration for
client/server Storm version mismatches, which I presume was the original
reasoning behind putting Storm functionality behind a Thrift API anyway.
And it would prevent crashed topologies during minor Storm version upgrades.

Reply via email to