I am using shuffelgrouping. With one worker it is running well. On Thu, May 14, 2015 at 4:18 PM, <[email protected]> wrote:
> *Dell - Internal Use - Confidential * > > Regarding the NotSerializableException: > > > > I would like to see what grouping you have defined between the two bolts > that you have experienced the NotSerializableException with. > > > > I am guessing it is localOrShuffleGrouping, can you change it to > shuffleGrouping and retest with one worker. > > > > *From:* Asif Ihsan [mailto:[email protected]] > *Sent:* Thursday, May 14, 2015 5:23 AM > *To:* [email protected] > *Subject:* Re: Multiple Workers in local mode gives error > > > > You are right about it. What does multiple workers means. Does it mean > that when I run topology with single worker than i will see one single > process handling all the bolts and when running 6 workers for 5 bolts and 1 > spout than i will see 6 separate processes running each bolt and spout? M i > right.If not then what does it mean? > > > > > > On Thu, May 14, 2015 at 2:46 PM, 임정택 <[email protected]> wrote: > > Hi. > > > > Storm serializes tuples when tuple should be sent to other (remote) worker. > > In other words, Storm doesn't serialize tuples when destination is local > task. That's why you didn't meet error when testing with 1 worker. > > > > MapEventBean seems to be not serializable, so you need to convert to other > data type. > > > > Hope this helps. > > > > Regards. > > Jungtaek Lim (HeartSaVioR) > > > > > > > > 2015-05-14 18:38 GMT+09:00 Asif Ihsan <[email protected]>: > > I am using Esper in one of the bolts of the Storm topology. Bolt emit > MapEventBean array (EventBean[]). With the single worker topology run > smoothly. With multiple workers it gives following error in emit function > call. > > > > May 14, 2015 2:37:07 PM clojure.tools.logging$eval1$fn__7 invoke > > SEVERE: Async loop died! > > java.lang.RuntimeException: java.lang.RuntimeException: > java.io.NotSerializableException: com.espertech.esper.event.map.MapEventBean > > at > backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:87) > > at > backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:58) > > at > backtype.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:62) > > at > backtype.storm.disruptor$consume_loop_STAR_$fn__1619.invoke(disruptor.clj:73) > > at backtype.storm.util$async_loop$fn__465.invoke(util.clj:377) > > at clojure.lang.AFn.run(AFn.java:24) > > at java.lang.Thread.run(Unknown Source) > > Caused by: java.lang.RuntimeException: java.io.NotSerializableException: > com.espertech.esper.event.map.MapEventBean > > at > backtype.storm.serialization.SerializableSerializer.write(SerializableSerializer.java:24) > > at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:554) > > at > com.esotericsoftware.kryo.serializers.MapSerializer.write(MapSerializer.java:87) > > at > com.esotericsoftware.kryo.serializers.MapSerializer.write(MapSerializer.java:17) > > at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:554) > > at > com.esotericsoftware.kryo.serializers.CollectionSerializer.write(CollectionSerializer.java:77) > > at > com.esotericsoftware.kryo.serializers.CollectionSerializer.write(CollectionSerializer.java:18) > > at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:472) > > at > backtype.storm.serialization.KryoValuesSerializer.serializeInto(KryoValuesSerializer.java:27) > > at > backtype.storm.serialization.KryoTupleSerializer.serialize(KryoTupleSerializer.java:27) > > at > backtype.storm.daemon.worker$mk_transfer_fn$fn__4152$fn__4156.invoke(worker.clj:99) > > at backtype.storm.util$fast_list_map.invoke(util.clj:771) > > at > backtype.storm.daemon.worker$mk_transfer_fn$fn__4152.invoke(worker.clj:99) > > at > backtype.storm.daemon.executor$start_batch_transfer__GT_worker_handler_BANG_$fn__3914.invoke(executor.clj:238) > > at > backtype.storm.disruptor$clojure_handler$reify__1606.onEvent(disruptor.clj:43) > > at > backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:84) > > ... 6 more > > Caused by: java.io.NotSerializableException: > com.espertech.esper.event.map.MapEventBean > > at java.io.ObjectOutputStream.writeObject0(Unknown Source) > > at java.io.ObjectOutputStream.writeArray(Unknown Source) > > at java.io.ObjectOutputStream.writeObject0(Unknown Source) > > at java.io.ObjectOutputStream.writeObject(Unknown Source) > > at > backtype.storm.serialization.SerializableSerializer.write(SerializableSerializer.java:21) > > ... 21 more > > > > May 14, 2015 2:37:07 PM clojure.tools.logging$eval1$fn__7 invoke > > SEVERE: > > java.lang.RuntimeException: java.lang.RuntimeException: > java.io.NotSerializableException: com.espertech.esper.event.map.MapEventBean > > at > backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:87) > > at > backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:58) > > at > backtype.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:62) > > at > backtype.storm.disruptor$consume_loop_STAR_$fn__1619.invoke(disruptor.clj:73) > > at backtype.storm.util$async_loop$fn__465.invoke(util.clj:377) > > at clojure.lang.AFn.run(AFn.java:24) > > at java.lang.Thread.run(Unknown Source) > > Caused by: java.lang.RuntimeException: java.io.NotSerializableException: > com.espertech.esper.event.map.MapEventBean > > at > backtype.storm.serialization.SerializableSerializer.write(SerializableSerializer.java:24) > > at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:554) > > at > com.esotericsoftware.kryo.serializers.MapSerializer.write(MapSerializer.java:87) > > at > com.esotericsoftware.kryo.serializers.MapSerializer.write(MapSerializer.java:17) > > at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:554) > > at > com.esotericsoftware.kryo.serializers.CollectionSerializer.write(CollectionSerializer.java:77) > > at > com.esotericsoftware.kryo.serializers.CollectionSerializer.write(CollectionSerializer.java:18) > > at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:472) > > at > backtype.storm.serialization.KryoValuesSerializer.serializeInto(KryoValuesSerializer.java:27) > > at > backtype.storm.serialization.KryoTupleSerializer.serialize(KryoTupleSerializer.java:27) > > at > backtype.storm.daemon.worker$mk_transfer_fn$fn__4152$fn__4156.invoke(worker.clj:99) > > at backtype.storm.util$fast_list_map.invoke(util.clj:771) > > at > backtype.storm.daemon.worker$mk_transfer_fn$fn__4152.invoke(worker.clj:99) > > at > backtype.storm.daemon.executor$start_batch_transfer__GT_worker_handler_BANG_$fn__3914.invoke(executor.clj:238) > > at > backtype.storm.disruptor$clojure_handler$reify__1606.onEvent(disruptor.clj:43) > > at > backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:84) > > ... 6 more > > Caused by: java.io.NotSerializableException: > com.espertech.esper.event.map.MapEventBean > > at java.io.ObjectOutputStream.writeObject0(Unknown Source) > > at java.io.ObjectOutputStream.writeArray(Unknown Source) > > at java.io.ObjectOutputStream.writeObject0(Unknown Source) > > at java.io.ObjectOutputStream.writeObject(Unknown Source) > > at > backtype.storm.serialization.SerializableSerializer.write(SerializableSerializer.java:21) > > ... 21 more > > > > > > On Wed, May 13, 2015 at 6:11 PM, Jeffery Maass <[email protected]> wrote: > > Asif: > > Would love to help you, however, we will need way more details. Why don't > you start with telling us what the error is? How are you detecting the > error? > > You will want to look over both the supervisor and worker logs. > > > Thank you for your time! > > +++++++++++++++++++++ > Jeff Maass <[email protected]> > linkedin.com/in/jeffmaass > stackoverflow.com/users/373418/maassql > +++++++++++++++++++++ > > > > On Wed, May 13, 2015 at 2:37 AM, Asif Ihsan <[email protected]> > wrote: > > Hi, > > > > I recently started using Storm. I am having topology. It works well with > numofworkers=1. However when I change it to 6 or more than 1. Its gives > error when emitting tuples. What is the issue and how to resolve it with > multiple workers. Thank you. > > > > -- > > Regards > > Muhammad Asif Ihsan > > > > > > > > -- > > Regards > > Muhammad Asif Ihsan > > > > > > -- > > Name : 임 정택 > Blog : http://www.heartsavior.net / http://dev.heartsavior.net > Twitter : http://twitter.com/heartsavior > > LinkedIn : http://www.linkedin.com/in/heartsavior > > > > > > -- > > Regards > > Muhammad Asif Ihsan > -- Regards Muhammad Asif Ihsan
