anyone here knows how to switch to zmq from netty? Just wanted to check that once before going down to 0.8.1.
On 10 July 2014 18:46, Suparno Datta <[email protected]> wrote: > @Stephan, Worked like a charm. How stupid of me not to change the local > directory. > > @Harsha. Didnt solve the original problem :( . > > Now getting this ones > > #.s.m.n.Client [INFO] Reconnect started for > Netty-Client-cluster1-fos-ThinkPad-T520/10.42.0.21:6700... [11] > > and after 30 retries finally the worker crashes > > 14-07-10 18:44:43 b.s.m.n.Client [INFO] Closing Netty Client > Netty-Client-cluster1-fos-ThinkPad-T520/10.42.0.21:6700 > 2014-07-10 18:44:43 b.s.m.n.Client [INFO] Waiting for pending batchs to be > sent with Netty-Client-cluster1-fos-ThinkPad-T520/10.42.0.21:6700..., > timeout: 600000ms, pendings: 0 > 2014-07-10 18:44:43 b.s.util [ERROR] Async loop died! > java.lang.RuntimeException: java.lang.RuntimeException: Client is being > closed, and does not take requests any more > at > backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:128) > ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating] > at > backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:99) > ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating] > at > backtype.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:80) > ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating] > at > backtype.storm.disruptor$consume_loop_STAR_$fn__758.invoke(disruptor.clj:94) > ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating] > at backtype.storm.util$async_loop$fn__457.invoke(util.clj:431) > ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating] > at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na] > at java.lang.Thread.run(Thread.java:744) [na:1.7.0_55] > Caused by: java.lang.RuntimeException: Client is being closed, and does > not take requests any more > at backtype.storm.messaging.netty.Client.send(Client.java:194) > ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating] > at backtype.storm.utils.TransferDrainer.send(TransferDrainer.java:54) > ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating] > at > backtype.storm.daemon.worker$mk_transfer_tuples_handler$fn__5927$fn__5928.invoke(worker.clj:322) > ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating] > at > backtype.storm.daemon.worker$mk_transfer_tuples_handler$fn__5927.invoke(worker.clj:320) > ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating] > at > backtype.storm.disruptor$clojure_handler$reify__745.onEvent(disruptor.clj:58) > ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating] > at > backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:125) > ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating] > ... 6 common frames omitted > 2014-07-10 18:44:43 b.s.util [INFO] Halting process: ("Async loop died!") > > Seems 0.8.1 it is. > > > On 10 July 2014 18:21, Kemper, Stephan <[email protected]> wrote: > >> We ran into this same problem this week. The problem isn't with >> ZooKeeper, but the local state files in your ${storm.local.dir}. If you >> delete the ./localstate directory there and restart the node, you should be >> OK again. >> >> More info on the problem was in last month's "v0.9.2-incubating and >> .ser files" thread from this mailing list. >> >> >> Stephan Kemper >> ViaSat >> >> From: Harsha <[email protected]> >> Reply-To: "[email protected]" < >> [email protected]> >> Date: Thursday, July 10, 2014 at 9:15 AM >> To: "[email protected]" <[email protected]> >> Subject: Re: b.s.m.n.Client [INFO] Reconnect >> >> Suparno, >> Old storm data in zookeeper might conflict with newer >> versions of storm. I would suggest you to bring down the topologies and >> clean zookeeper /storm dir. >> -Harsha >> >> >> >> On Thu, Jul 10, 2014, at 09:06 AM, Suparno Datta wrote: >> >> okay that got worse. I just downloaded the 0.9.2. and failed to launch >> the supervisors (nimbus is running though). You don't have to don any clean >> up before you launch the new version right ? >> >> Anyways the stack trace of the error >> >> 014-07-10 18:01:27 b.s.event [ERROR] Error when processing event >> java.lang.RuntimeException: java.io.InvalidClassException: >> clojure.lang.APersistentMap; local class incompatible: stream classdesc >> serialVersionUID = 270281984708184947, local class serialVersionUID = >> 8648225932767613808 >> at backtype.storm.utils.Utils.deserialize(Utils.java:93) >> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating] >> at backtype.storm.utils.LocalState.snapshot(LocalState.java:45) >> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating] >> at backtype.storm.utils.LocalState.get(LocalState.java:56) >> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating] >> at >> backtype.storm.daemon.supervisor$sync_processes.invoke(supervisor.clj:207) >> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating] >> at clojure.lang.AFn.applyToHelper(AFn.java:161) [clojure-1.5.1.jar:na] >> at clojure.lang.AFn.applyTo(AFn.java:151) [clojure-1.5.1.jar:na] >> at clojure.core$apply.invoke(core.clj:619) ~[clojure-1.5.1.jar:na] >> at clojure.core$partial$fn__4190.doInvoke(core.clj:2396) >> ~[clojure-1.5.1.jar:na] >> at clojure.lang.RestFn.invoke(RestFn.java:397) ~[clojure-1.5.1.jar:na] >> at backtype.storm.event$event_manager$fn__2378.invoke(event.clj:39) >> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating] >> at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na] >> at java.lang.Thread.run(Thread.java:744) [na:1.7.0_55] >> Caused by: java.io.InvalidClassException: clojure.lang.APersistentMap; >> local class incompatible: stream classdesc serialVersionUID = >> 270281984708184947, local class serialVersionUID = 8648225932767613808 >> at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:617) >> ~[na:1.7.0_55] >> at >> java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1622) >> ~[na:1.7.0_55] >> at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1517) >> ~[na:1.7.0_55] >> at >> java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1622) >> ~[na:1.7.0_55] >> at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1517) >> ~[na:1.7.0_55] >> at >> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771) >> ~[na:1.7.0_55] >> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350) >> ~[na:1.7.0_55] >> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370) >> ~[na:1.7.0_55] >> at java.util.HashMap.readObject(HashMap.java:1184) ~[na:1.7.0_55] >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> ~[na:1.7.0_55] >> at >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) >> ~[na:1.7.0_55] >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> ~[na:1.7.0_55] >> at java.lang.reflect.Method.invoke(Method.java:606) ~[na:1.7.0_55] >> at >> java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017) >> ~[na:1.7.0_55] >> at >> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1893) >> ~[na:1.7.0_55] >> at >> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798) >> ~[na:1.7.0_55] >> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350) >> ~[na:1.7.0_55] >> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370) >> ~[na:1.7.0_55] >> at backtype.storm.utils.Utils.deserialize(Utils.java:89) >> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating] >> ... 11 common frames omitted >> >> >> On 10 July 2014 17:20, Harsha <[email protected]> wrote: >> >> >> Yes. As per the change log >> https://github.com/apache/incubator-storm/blob/v0.9.2-incubating/CHANGELOG.md >> STORM-187 >> did make the 0.9.2 release. >> >> >> On Thu, Jul 10, 2014, at 08:11 AM, Suparno Datta wrote: >> >> You think it's fixed in 0.9.2 ? >> >> >> >> On 10 July 2014 17:08, Suparno Datta <[email protected]> wrote: >> >> I just found that too. Seems it's becuse 0.9.1 usses netty by default >> instead of zeromq ( guess thats why it was working with 0.8.1). Presently >> looking for the configuration parameter by which i can tell it to use zmq >> instead of netty. Let me know if you have any clue. Otherwise i just have >> to chuck the 0.9.1 and get back to 0.8.1 >> >> >> On 10 July 2014 17:02, Harsha <[email protected]> wrote: >> >> >> Hi Suparno, >> It might be because of >> https://issues.apache.org/jira/browse/STORM-187. Can you try using >> 0.9.2-incubating release. >> -Harsha >> >> >> >> On Thu, Jul 10, 2014, at 07:38 AM, Suparno Datta wrote: >> >> Hi, >> >> I am using storm 0.9.1-incubating on a single machine cluster to run a >> simple twitter hashtag extractor. I am using the Storm-twitter-workshop >> which i found to be extremely useful. >> >> https://github.com/kantega/storm-twitter-workshop >> >> I have used this program before with storm 0.8.1 and it ran like a charm. >> I might mention that was on a server machine with quite 2 quad xeon >> processors. >> >> This time i am trying it on my laptop( i5 , 8GB). But i am constantly >> getting this error in the worker log files >> >> 2014-07-10 13:01:47 b.s.m.n.Client [INFO] Reconnect ... [24] >> 2014-07-10 13:01:58 b.s.m.n.Client [INFO] Reconnect ... [25] >> 2014-07-10 13:02:09 b.s.m.n.Client [INFO] Reconnect ... [26] >> 2014-07-10 13:02:19 STDIO [ERROR] Jul 10, 2014 1:02:19 PM >> org.jboss.netty.channel.DefaultChannelPipeline >> WARNING: An exception was thrown by a user handler while handling an >> exception event ([id: 0x563f7062] EXCEPTION: java.net.ConnectException: >> connection timed out) >> java.lang.IllegalArgumentException: timeout value is negative >> at java.lang.Thread.sleep(Native Method) >> at backtype.storm.messaging.netty.Client.reconnect(Client.java:94) >> at >> backtype.storm.messaging.netty.StormClientHandler.exceptionCaught(StormClientHandler.java:118) >> at >> org.jboss.netty.handler.codec.frame.FrameDecoder.exceptionCaught(FrameDecoder.java:377) >> at >> org.jboss.netty.channel.Channels.fireExceptionCaught(Channels.java:525) >> at >> org.jboss.netty.channel.socket.nio.NioClientBoss.processConnectTimeout(NioClientBoss.java:140) >> at >> org.jboss.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:82) >> at >> org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:312) >> at >> org.jboss.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:41) >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >> at java.lang.Thread.run(Thread.java:744) >> >> >> Now here comes the stranges part. If i declare just one instance of the >> hashtag extractor Bolt it fails to get anything but for more that 1 it does >> manage to get me a few hashtags though with quite high latency. Another >> strange part related to this machine is if i declare more than 2 >> supervisor.slots.ports the program doesnt even launch any more showing some >> initialitzation error. >> >> Sorry if i blabbered a lot about the hardware and stuff. But somehow to >> me it seemed quite related to the problem. Any sort of help will be really >> useful. >> >> Thanks, >> >> Suparno >> >> >> >> >> >> >> >> -- >> Suparno Datta >> >> >> >> >> -- >> Suparno Datta >> >> >> >> >> >> >> -- >> Suparno Datta >> >> >> > > > > -- > Suparno Datta > -- Suparno Datta
