Look at "worker.childopts" and "worker.heap.size.mb" in storm.yaml on the nimbus node. Try increasing worker's heap size by specifying JVM opt "-Xmx <heap size>".
2015-11-24 10:59 GMT-05:00 prakash a <[email protected]>: > We are getting below error from kafka spout on our storm cluster.Please >> let us know which configuration need to be updated. >> >> >> ------------------------------------------------------------------------------------------------------------------ >> >> java.lang.OutOfMemoryError: Java heap space >> at java.util.Arrays.copyOf(Arrays.java:2271) >> at >> java.io.ByteArrayOutputStream.toByteArray(ByteArrayOutputStream.java:178) >> at >> kafka.message.ByteBufferMessageSet$.decompress(ByteBufferMessageSet.scala:75) >> at >> kafka.message.ByteBufferMessageSet$$anon$1.makeNextOuter(ByteBufferMessageSet.scala:178) >> at >> kafka.message.ByteBufferMessageSet$$anon$1.makeNext(ByteBufferMessageSet.scala:191) >> at >> kafka.message.ByteBufferMessageSet$$anon$1.makeNext(ByteBufferMessageSet.scala:145) >> at >> kafka.utils.IteratorTemplate.maybeComputeNext(IteratorTemplate.scala:66) >> at kafka.utils.IteratorTemplate.hasNext(IteratorTemplate.scala:58) >> at >> kafka.javaapi.message.ByteBufferMessageSet$$anon$1.hasNext(ByteBufferMessageSet.scala:42) >> at storm.kafka.PartitionManager.fill(PartitionManager.java:177) >> at storm.kafka.PartitionManager.next(PartitionManager.java:124) >> at storm.kafka.KafkaSpout.nextTuple(KafkaSpout.java:141) >> at >> backtype.storm.daemon.executor$fn__3231$fn__3246$fn__3275.invoke(executor.clj:562) >> at backtype.storm.util$async_loop$fn__442.invoke(util.clj:436) >> at clojure.lang.AFn.run(AFn.java:24) >> at java.lang.Thread.run(Thread.java:744) >> >> >> Kafka version 0.8.1.1 >> Storm 0.9.1.2.1.11.0-891 >> >> -- >> Regards, >> Prakash. >> --------------------------------------------------------- >> Doing your best is more important than being the best. >> ---------------------------------------------------------- >> > > > > -- > Regards, > Prakash. > --------------------------------------------------------- > Doing your best is more important than being the best. > ---------------------------------------------------------- > -- Sincerely, Fan Jiang
