We ran into this same problem this week. The problem isn't with ZooKeeper, but
the local state files in your ${storm.local.dir}. If you delete the
./localstate directory there and restart the node, you should be OK again.
More info on the problem was in last month's "v0.9.2-incubating and .ser files"
thread from this mailing list.
Stephan Kemper
ViaSat
From: Harsha <[email protected]<mailto:[email protected]>>
Reply-To:
"[email protected]<mailto:[email protected]>"
<[email protected]<mailto:[email protected]>>
Date: Thursday, July 10, 2014 at 9:15 AM
To: "[email protected]<mailto:[email protected]>"
<[email protected]<mailto:[email protected]>>
Subject: Re: b.s.m.n.Client [INFO] Reconnect
Suparno,
Old storm data in zookeeper might conflict with newer versions of
storm. I would suggest you to bring down the topologies and clean zookeeper
/storm dir.
-Harsha
On Thu, Jul 10, 2014, at 09:06 AM, Suparno Datta wrote:
okay that got worse. I just downloaded the 0.9.2. and failed to launch the
supervisors (nimbus is running though). You don't have to don any clean up
before you launch the new version right ?
Anyways the stack trace of the error
014-07-10 18:01:27 b.s.event [ERROR] Error when processing event
java.lang.RuntimeException: java.io.InvalidClassException:
clojure.lang.APersistentMap; local class incompatible: stream classdesc
serialVersionUID = 270281984708184947, local class serialVersionUID =
8648225932767613808
at backtype.storm.utils.Utils.deserialize(Utils.java:93)
~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at backtype.storm.utils.LocalState.snapshot(LocalState.java:45)
~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at backtype.storm.utils.LocalState.get(LocalState.java:56)
~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at backtype.storm.daemon.supervisor$sync_processes.invoke(supervisor.clj:207)
~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at clojure.lang.AFn.applyToHelper(AFn.java:161) [clojure-1.5.1.jar:na]
at clojure.lang.AFn.applyTo(AFn.java:151) [clojure-1.5.1.jar:na]
at clojure.core$apply.invoke(core.clj:619) ~[clojure-1.5.1.jar:na]
at clojure.core$partial$fn__4190.doInvoke(core.clj:2396) ~[clojure-1.5.1.jar:na]
at clojure.lang.RestFn.invoke(RestFn.java:397) ~[clojure-1.5.1.jar:na]
at backtype.storm.event$event_manager$fn__2378.invoke(event.clj:39)
~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na]
at java.lang.Thread.run(Thread.java:744) [na:1.7.0_55]
Caused by: java.io.InvalidClassException: clojure.lang.APersistentMap; local
class incompatible: stream classdesc serialVersionUID = 270281984708184947,
local class serialVersionUID = 8648225932767613808
at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:617)
~[na:1.7.0_55]
at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1622)
~[na:1.7.0_55]
at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1517)
~[na:1.7.0_55]
at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1622)
~[na:1.7.0_55]
at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1517)
~[na:1.7.0_55]
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771)
~[na:1.7.0_55]
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
~[na:1.7.0_55]
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
~[na:1.7.0_55]
at java.util.HashMap.readObject(HashMap.java:1184) ~[na:1.7.0_55]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.7.0_55]
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
~[na:1.7.0_55]
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[na:1.7.0_55]
at java.lang.reflect.Method.invoke(Method.java:606) ~[na:1.7.0_55]
at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017)
~[na:1.7.0_55]
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1893)
~[na:1.7.0_55]
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
~[na:1.7.0_55]
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
~[na:1.7.0_55]
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
~[na:1.7.0_55]
at backtype.storm.utils.Utils.deserialize(Utils.java:89)
~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
... 11 common frames omitted
On 10 July 2014 17:20, Harsha <[email protected]<mailto:[email protected]>> wrote:
Yes. As per the change log
https://github.com/apache/incubator-storm/blob/v0.9.2-incubating/CHANGELOG.md
STORM-187 did make the 0.9.2 release.
On Thu, Jul 10, 2014, at 08:11 AM, Suparno Datta wrote:
You think it's fixed in 0.9.2 ?
On 10 July 2014 17:08, Suparno Datta
<[email protected]<mailto:[email protected]>> wrote:
I just found that too. Seems it's becuse 0.9.1 usses netty by default instead
of zeromq ( guess thats why it was working with 0.8.1). Presently looking for
the configuration parameter by which i can tell it to use zmq instead of netty.
Let me know if you have any clue. Otherwise i just have to chuck the 0.9.1 and
get back to 0.8.1
On 10 July 2014 17:02, Harsha <[email protected]<mailto:[email protected]>> wrote:
Hi Suparno,
It might be because of https://issues.apache.org/jira/browse/STORM-187.
Can you try using 0.9.2-incubating release.
-Harsha
On Thu, Jul 10, 2014, at 07:38 AM, Suparno Datta wrote:
Hi,
I am using storm 0.9.1-incubating on a single machine cluster to run a simple
twitter hashtag extractor. I am using the Storm-twitter-workshop which i found
to be extremely useful.
https://github.com/kantega/storm-twitter-workshop
I have used this program before with storm 0.8.1 and it ran like a charm. I
might mention that was on a server machine with quite 2 quad xeon processors.
This time i am trying it on my laptop( i5 , 8GB). But i am constantly getting
this error in the worker log files
2014-07-10 13:01:47 b.s.m.n.Client [INFO] Reconnect ... [24]
2014-07-10 13:01:58 b.s.m.n.Client [INFO] Reconnect ... [25]
2014-07-10 13:02:09 b.s.m.n.Client [INFO] Reconnect ... [26]
2014-07-10 13:02:19 STDIO [ERROR] Jul 10, 2014 1:02:19 PM
org.jboss.netty.channel.DefaultChannelPipeline
WARNING: An exception was thrown by a user handler while handling an exception
event ([id: 0x563f7062] EXCEPTION: java.net.ConnectException: connection timed
out)
java.lang.IllegalArgumentException: timeout value is negative
at java.lang.Thread.sleep(Native Method)
at backtype.storm.messaging.netty.Client.reconnect(Client.java:94)
at
backtype.storm.messaging.netty.StormClientHandler.exceptionCaught(StormClientHandler.java:118)
at
org.jboss.netty.handler.codec.frame.FrameDecoder.exceptionCaught(FrameDecoder.java:377)
at org.jboss.netty.channel.Channels.fireExceptionCaught(Channels.java:525)
at
org.jboss.netty.channel.socket.nio.NioClientBoss.processConnectTimeout(NioClientBoss.java:140)
at
org.jboss.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:82)
at
org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:312)
at org.jboss.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:41)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Now here comes the stranges part. If i declare just one instance of the hashtag
extractor Bolt it fails to get anything but for more that 1 it does manage to
get me a few hashtags though with quite high latency. Another strange part
related to this machine is if i declare more than 2 supervisor.slots.ports the
program doesnt even launch any more showing some initialitzation error.
Sorry if i blabbered a lot about the hardware and stuff. But somehow to me it
seemed quite related to the problem. Any sort of help will be really useful.
Thanks,
Suparno
--
Suparno Datta
--
Suparno Datta
--
Suparno Datta
