I am using Esper in one of the bolts of the Storm topology. Bolt emit MapEventBean array (EventBean[]). With the single worker topology run smoothly. With multiple workers it gives following error in emit function call.
May 14, 2015 2:37:07 PM clojure.tools.logging$eval1$fn__7 invoke SEVERE: Async loop died! java.lang.RuntimeException: java.lang.RuntimeException: java.io.NotSerializableException: com.espertech.esper.event.map.MapEventBean at backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:87) at backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:58) at backtype.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:62) at backtype.storm.disruptor$consume_loop_STAR_$fn__1619.invoke(disruptor.clj:73) at backtype.storm.util$async_loop$fn__465.invoke(util.clj:377) at clojure.lang.AFn.run(AFn.java:24) at java.lang.Thread.run(Unknown Source) Caused by: java.lang.RuntimeException: java.io.NotSerializableException: com.espertech.esper.event.map.MapEventBean at backtype.storm.serialization.SerializableSerializer.write(SerializableSerializer.java:24) at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:554) at com.esotericsoftware.kryo.serializers.MapSerializer.write(MapSerializer.java:87) at com.esotericsoftware.kryo.serializers.MapSerializer.write(MapSerializer.java:17) at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:554) at com.esotericsoftware.kryo.serializers.CollectionSerializer.write(CollectionSerializer.java:77) at com.esotericsoftware.kryo.serializers.CollectionSerializer.write(CollectionSerializer.java:18) at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:472) at backtype.storm.serialization.KryoValuesSerializer.serializeInto(KryoValuesSerializer.java:27) at backtype.storm.serialization.KryoTupleSerializer.serialize(KryoTupleSerializer.java:27) at backtype.storm.daemon.worker$mk_transfer_fn$fn__4152$fn__4156.invoke(worker.clj:99) at backtype.storm.util$fast_list_map.invoke(util.clj:771) at backtype.storm.daemon.worker$mk_transfer_fn$fn__4152.invoke(worker.clj:99) at backtype.storm.daemon.executor$start_batch_transfer__GT_worker_handler_BANG_$fn__3914.invoke(executor.clj:238) at backtype.storm.disruptor$clojure_handler$reify__1606.onEvent(disruptor.clj:43) at backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:84) ... 6 more Caused by: java.io.NotSerializableException: com.espertech.esper.event.map.MapEventBean at java.io.ObjectOutputStream.writeObject0(Unknown Source) at java.io.ObjectOutputStream.writeArray(Unknown Source) at java.io.ObjectOutputStream.writeObject0(Unknown Source) at java.io.ObjectOutputStream.writeObject(Unknown Source) at backtype.storm.serialization.SerializableSerializer.write(SerializableSerializer.java:21) ... 21 more May 14, 2015 2:37:07 PM clojure.tools.logging$eval1$fn__7 invoke SEVERE: java.lang.RuntimeException: java.lang.RuntimeException: java.io.NotSerializableException: com.espertech.esper.event.map.MapEventBean at backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:87) at backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:58) at backtype.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:62) at backtype.storm.disruptor$consume_loop_STAR_$fn__1619.invoke(disruptor.clj:73) at backtype.storm.util$async_loop$fn__465.invoke(util.clj:377) at clojure.lang.AFn.run(AFn.java:24) at java.lang.Thread.run(Unknown Source) Caused by: java.lang.RuntimeException: java.io.NotSerializableException: com.espertech.esper.event.map.MapEventBean at backtype.storm.serialization.SerializableSerializer.write(SerializableSerializer.java:24) at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:554) at com.esotericsoftware.kryo.serializers.MapSerializer.write(MapSerializer.java:87) at com.esotericsoftware.kryo.serializers.MapSerializer.write(MapSerializer.java:17) at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:554) at com.esotericsoftware.kryo.serializers.CollectionSerializer.write(CollectionSerializer.java:77) at com.esotericsoftware.kryo.serializers.CollectionSerializer.write(CollectionSerializer.java:18) at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:472) at backtype.storm.serialization.KryoValuesSerializer.serializeInto(KryoValuesSerializer.java:27) at backtype.storm.serialization.KryoTupleSerializer.serialize(KryoTupleSerializer.java:27) at backtype.storm.daemon.worker$mk_transfer_fn$fn__4152$fn__4156.invoke(worker.clj:99) at backtype.storm.util$fast_list_map.invoke(util.clj:771) at backtype.storm.daemon.worker$mk_transfer_fn$fn__4152.invoke(worker.clj:99) at backtype.storm.daemon.executor$start_batch_transfer__GT_worker_handler_BANG_$fn__3914.invoke(executor.clj:238) at backtype.storm.disruptor$clojure_handler$reify__1606.onEvent(disruptor.clj:43) at backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:84) ... 6 more Caused by: java.io.NotSerializableException: com.espertech.esper.event.map.MapEventBean at java.io.ObjectOutputStream.writeObject0(Unknown Source) at java.io.ObjectOutputStream.writeArray(Unknown Source) at java.io.ObjectOutputStream.writeObject0(Unknown Source) at java.io.ObjectOutputStream.writeObject(Unknown Source) at backtype.storm.serialization.SerializableSerializer.write(SerializableSerializer.java:21) ... 21 more On Wed, May 13, 2015 at 6:11 PM, Jeffery Maass <[email protected]> wrote: > Asif: > > Would love to help you, however, we will need way more details. Why don't > you start with telling us what the error is? How are you detecting the > error? > > You will want to look over both the supervisor and worker logs. > > Thank you for your time! > > +++++++++++++++++++++ > Jeff Maass <[email protected]> > linkedin.com/in/jeffmaass > stackoverflow.com/users/373418/maassql > +++++++++++++++++++++ > > > On Wed, May 13, 2015 at 2:37 AM, Asif Ihsan <[email protected]> > wrote: > >> Hi, >> >> I recently started using Storm. I am having topology. It works well with >> numofworkers=1. However when I change it to 6 or more than 1. Its gives >> error when emitting tuples. What is the issue and how to resolve it with >> multiple workers. Thank you. >> >> -- >> Regards >> Muhammad Asif Ihsan >> > > -- Regards Muhammad Asif Ihsan
