It can work with ZMQ, but you MUST use the version specified (2.1.7). Newer versions change the API which causes errors, which might be what you are seeing. Is the version of libzmq you installed the same as the one you are using in production?
On Fri, Jan 31, 2014 at 9:47 AM, Mark Greene <[email protected]> wrote: > Storm uses the internal queuing (through ZMQ) only when there is a >> communication between two worker processes is required,which is why this >> error comes up only when you set num_workers>1. > > > I'm a little confused by the answer, are you suggesting that storm cannot > run more than one worker even with the correct (older) version of ZMQ? > > What's unique about the environment I was having trouble with is it only > had 1 supervisor where as my prod environment has multiple supervisors and > I am not seeing a problem there. > > > On Thu, Jan 30, 2014 at 11:59 PM, bijoy deb <[email protected]>wrote: > >> Hi Mark, >> >> Storm uses the internal queuing (through ZMQ) only when there is a >> communication between two worker processes is required,which is why this >> error comes up only when you set num_workers>1. >> >> Though I won't be able to help with with an exact solution for this,I can >> provide some pointers: >> >> a) Regarding the reason for the error,documentation says that The >> ZMQ/zeromq version needs to be downgraded to 2.1.7 if its higher than that. >> b) In a future version of Storm (don't recollect the exact version number >> or if it has already been released),they are supposed to remove the ZMQ >> dependency at all,so the above error should not be coming then. >> >> Thanks >> Bijoy >> >> >> On Fri, Jan 31, 2014 at 8:42 AM, Mark Greene <[email protected]> wrote: >> >>> Exception in log: >>> >>> 2014-01-31 02:58:14 task [INFO] Emitting: change-spout default >>> [[B@38fc659c] >>> 2014-01-31 02:58:14 task [INFO] Emitting: change-spout __ack_init >>> [1863657906985036001 0 2] >>> 2014-01-31 02:58:14 util [ERROR] Async loop died! >>> java.lang.RuntimeException: org.zeromq.ZMQException: Invalid >>> argument(0x16) >>> at >>> backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:87) >>> at >>> backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:58) >>> at >>> backtype.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:62) >>> at >>> backtype.storm.disruptor$consume_loop_STAR_$fn__1619.invoke(disruptor.clj:73) >>> at backtype.storm.util$async_loop$fn__465.invoke(util.clj:377) >>> at clojure.lang.AFn.run(AFn.java:24) >>> at java.lang.Thread.run(Thread.java:744) >>> Caused by: org.zeromq.ZMQException: Invalid argument(0x16) >>> at org.zeromq.ZMQ$Socket.send(Native Method) >>> at zilch.mq$send.invoke(mq.clj:93) >>> at backtype.storm.messaging.zmq.ZMQConnection.send(zmq.clj:43) >>> at >>> backtype.storm.daemon.worker$mk_transfer_tuples_handler$fn__4333$fn__4334.invoke(worker.clj:298) >>> at >>> backtype.storm.daemon.worker$mk_transfer_tuples_handler$fn__4333.invoke(worker.clj:287) >>> at >>> backtype.storm.disruptor$clojure_handler$reify__1606.onEvent(disruptor.clj:43) >>> at >>> backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:84) >>> ... 6 more >>> 2014-01-31 02:58:14 util [INFO] Halting process: ("Async loop died!") >>> 2014-01-31 02:58:24 executor [INFO] Processing received message source: >>> __system:-1, stream: __tick, id: {}, [30] >>> >>> I see the above exception almost immediately upon which my spout emits >>> the first tuple from the queue. I have pared down my topology so there is >>> just one spout and no bolts so as to narrow the problem down but the only >>> time I can keep the spout running is if I omit the collector.emit call >>> itself. >>> >>> I'm not sure if it would make a difference but the supervisor has three >>> slots and this topology would occupy two of them, however, when configured >>> with two I get the above exception, when configured with one, everything >>> works fine. >>> >>> >>> >> >
