You might want to see if tuples are backing up and eating up memory, or if one
of your components in the worker is holding on to stale references, and
preventing them from being reaped.
You can add the -XX:HeapDumpPath=/path/to/heapdumps
-XX:HeapDumpOnOutOfMemoryError arguments to your worker childopts so that you
can see what your worker was holding on to.
On Tuesday, November 24, 2015 10:09 AM, Fan Jiang <[email protected]> wrote:
Look at "worker.childopts" and "worker.heap.size.mb" in storm.yaml on the
nimbus node. Try increasing worker's heap size by specifying JVM opt "-Xmx
<heap size>".
2015-11-24 10:59 GMT-05:00 prakash a <[email protected]>:
We are getting below error from kafka spout on our storm cluster.Please let us
know which configuration need to be updated.
------------------------------------------------------------------------------------------------------------------java.lang.OutOfMemoryError:
Java heap space
at java.util.Arrays.copyOf(Arrays.java:2271)
at
java.io.ByteArrayOutputStream.toByteArray(ByteArrayOutputStream.java:178)
at
kafka.message.ByteBufferMessageSet$.decompress(ByteBufferMessageSet.scala:75)
at
kafka.message.ByteBufferMessageSet$$anon$1.makeNextOuter(ByteBufferMessageSet.scala:178)
at
kafka.message.ByteBufferMessageSet$$anon$1.makeNext(ByteBufferMessageSet.scala:191)
at
kafka.message.ByteBufferMessageSet$$anon$1.makeNext(ByteBufferMessageSet.scala:145)
at
kafka.utils.IteratorTemplate.maybeComputeNext(IteratorTemplate.scala:66)
at kafka.utils.IteratorTemplate.hasNext(IteratorTemplate.scala:58)
at
kafka.javaapi.message.ByteBufferMessageSet$$anon$1.hasNext(ByteBufferMessageSet.scala:42)
at storm.kafka.PartitionManager.fill(PartitionManager.java:177)
at storm.kafka.PartitionManager.next(PartitionManager.java:124)
at storm.kafka.KafkaSpout.nextTuple(KafkaSpout.java:141)
at
backtype.storm.daemon.executor$fn__3231$fn__3246$fn__3275.invoke(executor.clj:562)
at backtype.storm.util$async_loop$fn__442.invoke(util.clj:436)
at clojure.lang.AFn.run(AFn.java:24)
at java.lang.Thread.run(Thread.java:744)
Kafka version 0.8.1.1Storm 0.9.1.2.1.11.0-891
--
Regards,
Prakash.
---------------------------------------------------------
Doing your best is more important than being the best.
----------------------------------------------------------
--
Regards,
Prakash.
---------------------------------------------------------
Doing your best is more important than being the best.
----------------------------------------------------------
--
Sincerely,
Fan Jiang
