Hi, I am using Storm 1.0.2 My configuration is quite simple: `kafka-spout` feeding to `solr-bolt`
topology.workers = 2 spout.parallelism = 1 bolt.parallelism = 1 Our messages coming from kafka are large: around 100kb per message to max of 500kb per message. But I see lots of errors: Window Emitted Transferred Complete latency (ms) Acked Failed 10m 0s 355,160 355,161 15,263 29,040 340,823 And after running for 30 minutes, the kafka-spout goes OutOfMemory: java.lang.OutOfMemoryError: GC overhead limit exceeded at org.apache.storm.kafka.PartitionManager.fail(PartitionManager.java:281) at org.apache.storm.kafka.KafkaSpout.fail(KafkaSpout.java:173) at org.apache.storm.daemon.executor$fail_spout_msg.invoke(executor.clj:439) at org.apache.storm.daemon.executor$fn$reify__7993.expire(executor.clj:512) at org.apache.storm.utils.RotatingMap.rotate(RotatingMap.java:77) at org.apache.storm.daemon.executor$fn__7990$tuple_action_fn__7996.invoke(executor.clj:517) at org.apache.storm.daemon.executor$mk_task_receiver$fn__7979.invoke(executor.clj:467) at org.apache.storm.disruptor$clojure_handler$reify__7492.onEvent(disruptor.clj:40) at org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:451) at org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:430) at org.apache.storm.utils.DisruptorQueue.consumeBatch(DisruptorQueue.java:420) at org.apache.storm.disruptor$consume_batch.invoke(disruptor.clj:69) at org.apache.storm.daemon.executor$fn__7990$fn__8005$fn__8036.invoke(executor.clj:628) at org.apache.storm.util$async_loop$fn__624.invoke(util.clj:484) at clojure.lang.AFn.run(AFn.java:22) at java.lang.Thread.run(Thread.java:745) In the worker.log I see lots of ERRORs like (just in a duration of 30 minutes): java.nio.channels.ClosedChannelException (24957 times) java.net.ConnectException (107 times) java.io.IOException (22 times) How do I debug this? Thanks SG
