You are right about it. What does multiple workers means. Does it mean that when I run topology with single worker than i will see one single process handling all the bolts and when running 6 workers for 5 bolts and 1 spout than i will see 6 separate processes running each bolt and spout? M i right.If not then what does it mean?
On Thu, May 14, 2015 at 2:46 PM, 임정택 <[email protected]> wrote: > Hi. > > Storm serializes tuples when tuple should be sent to other (remote) worker. > In other words, Storm doesn't serialize tuples when destination is local > task. That's why you didn't meet error when testing with 1 worker. > > MapEventBean seems to be not serializable, so you need to convert to other > data type. > > Hope this helps. > > Regards. > Jungtaek Lim (HeartSaVioR) > > > > 2015-05-14 18:38 GMT+09:00 Asif Ihsan <[email protected]>: > >> I am using Esper in one of the bolts of the Storm topology. Bolt emit >> MapEventBean array (EventBean[]). With the single worker topology run >> smoothly. With multiple workers it gives following error in emit function >> call. >> >> May 14, 2015 2:37:07 PM clojure.tools.logging$eval1$fn__7 invoke >> SEVERE: Async loop died! >> java.lang.RuntimeException: java.lang.RuntimeException: >> java.io.NotSerializableException: com.espertech.esper.event.map.MapEventBean >> at >> backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:87) >> at >> backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:58) >> at >> backtype.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:62) >> at >> backtype.storm.disruptor$consume_loop_STAR_$fn__1619.invoke(disruptor.clj:73) >> at backtype.storm.util$async_loop$fn__465.invoke(util.clj:377) >> at clojure.lang.AFn.run(AFn.java:24) >> at java.lang.Thread.run(Unknown Source) >> Caused by: java.lang.RuntimeException: java.io.NotSerializableException: >> com.espertech.esper.event.map.MapEventBean >> at >> backtype.storm.serialization.SerializableSerializer.write(SerializableSerializer.java:24) >> at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:554) >> at >> com.esotericsoftware.kryo.serializers.MapSerializer.write(MapSerializer.java:87) >> at >> com.esotericsoftware.kryo.serializers.MapSerializer.write(MapSerializer.java:17) >> at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:554) >> at >> com.esotericsoftware.kryo.serializers.CollectionSerializer.write(CollectionSerializer.java:77) >> at >> com.esotericsoftware.kryo.serializers.CollectionSerializer.write(CollectionSerializer.java:18) >> at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:472) >> at >> backtype.storm.serialization.KryoValuesSerializer.serializeInto(KryoValuesSerializer.java:27) >> at >> backtype.storm.serialization.KryoTupleSerializer.serialize(KryoTupleSerializer.java:27) >> at >> backtype.storm.daemon.worker$mk_transfer_fn$fn__4152$fn__4156.invoke(worker.clj:99) >> at backtype.storm.util$fast_list_map.invoke(util.clj:771) >> at >> backtype.storm.daemon.worker$mk_transfer_fn$fn__4152.invoke(worker.clj:99) >> at >> backtype.storm.daemon.executor$start_batch_transfer__GT_worker_handler_BANG_$fn__3914.invoke(executor.clj:238) >> at >> backtype.storm.disruptor$clojure_handler$reify__1606.onEvent(disruptor.clj:43) >> at >> backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:84) >> ... 6 more >> Caused by: java.io.NotSerializableException: >> com.espertech.esper.event.map.MapEventBean >> at java.io.ObjectOutputStream.writeObject0(Unknown Source) >> at java.io.ObjectOutputStream.writeArray(Unknown Source) >> at java.io.ObjectOutputStream.writeObject0(Unknown Source) >> at java.io.ObjectOutputStream.writeObject(Unknown Source) >> at >> backtype.storm.serialization.SerializableSerializer.write(SerializableSerializer.java:21) >> ... 21 more >> >> May 14, 2015 2:37:07 PM clojure.tools.logging$eval1$fn__7 invoke >> SEVERE: >> java.lang.RuntimeException: java.lang.RuntimeException: >> java.io.NotSerializableException: com.espertech.esper.event.map.MapEventBean >> at >> backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:87) >> at >> backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:58) >> at >> backtype.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:62) >> at >> backtype.storm.disruptor$consume_loop_STAR_$fn__1619.invoke(disruptor.clj:73) >> at backtype.storm.util$async_loop$fn__465.invoke(util.clj:377) >> at clojure.lang.AFn.run(AFn.java:24) >> at java.lang.Thread.run(Unknown Source) >> Caused by: java.lang.RuntimeException: java.io.NotSerializableException: >> com.espertech.esper.event.map.MapEventBean >> at >> backtype.storm.serialization.SerializableSerializer.write(SerializableSerializer.java:24) >> at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:554) >> at >> com.esotericsoftware.kryo.serializers.MapSerializer.write(MapSerializer.java:87) >> at >> com.esotericsoftware.kryo.serializers.MapSerializer.write(MapSerializer.java:17) >> at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:554) >> at >> com.esotericsoftware.kryo.serializers.CollectionSerializer.write(CollectionSerializer.java:77) >> at >> com.esotericsoftware.kryo.serializers.CollectionSerializer.write(CollectionSerializer.java:18) >> at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:472) >> at >> backtype.storm.serialization.KryoValuesSerializer.serializeInto(KryoValuesSerializer.java:27) >> at >> backtype.storm.serialization.KryoTupleSerializer.serialize(KryoTupleSerializer.java:27) >> at >> backtype.storm.daemon.worker$mk_transfer_fn$fn__4152$fn__4156.invoke(worker.clj:99) >> at backtype.storm.util$fast_list_map.invoke(util.clj:771) >> at >> backtype.storm.daemon.worker$mk_transfer_fn$fn__4152.invoke(worker.clj:99) >> at >> backtype.storm.daemon.executor$start_batch_transfer__GT_worker_handler_BANG_$fn__3914.invoke(executor.clj:238) >> at >> backtype.storm.disruptor$clojure_handler$reify__1606.onEvent(disruptor.clj:43) >> at >> backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:84) >> ... 6 more >> Caused by: java.io.NotSerializableException: >> com.espertech.esper.event.map.MapEventBean >> at java.io.ObjectOutputStream.writeObject0(Unknown Source) >> at java.io.ObjectOutputStream.writeArray(Unknown Source) >> at java.io.ObjectOutputStream.writeObject0(Unknown Source) >> at java.io.ObjectOutputStream.writeObject(Unknown Source) >> at >> backtype.storm.serialization.SerializableSerializer.write(SerializableSerializer.java:21) >> ... 21 more >> >> >> On Wed, May 13, 2015 at 6:11 PM, Jeffery Maass <[email protected]> wrote: >> >>> Asif: >>> >>> Would love to help you, however, we will need way more details. Why >>> don't you start with telling us what the error is? How are you detecting >>> the error? >>> >>> You will want to look over both the supervisor and worker logs. >>> >>> Thank you for your time! >>> >>> +++++++++++++++++++++ >>> Jeff Maass <[email protected]> >>> linkedin.com/in/jeffmaass >>> stackoverflow.com/users/373418/maassql >>> +++++++++++++++++++++ >>> >>> >>> On Wed, May 13, 2015 at 2:37 AM, Asif Ihsan <[email protected]> >>> wrote: >>> >>>> Hi, >>>> >>>> I recently started using Storm. I am having topology. It works well >>>> with numofworkers=1. However when I change it to 6 or more than 1. Its >>>> gives error when emitting tuples. What is the issue and how to resolve it >>>> with multiple workers. Thank you. >>>> >>>> -- >>>> Regards >>>> Muhammad Asif Ihsan >>>> >>> >>> >> >> >> -- >> Regards >> Muhammad Asif Ihsan >> > > > > -- > Name : 임 정택 > Blog : http://www.heartsavior.net / http://dev.heartsavior.net > Twitter : http://twitter.com/heartsavior > LinkedIn : http://www.linkedin.com/in/heartsavior > -- Regards Muhammad Asif Ihsan
