You need to increase worker's Heap Size by adding the following in each
worker's storm.yaml file:
worker.childopts: "-Xmx16g"
Then restart each worker's supervisor.
On Wed, Mar 23, 2016 at 4:07 AM, Sai Dilip Reddy Kiralam <
[email protected]> wrote:
> Hi,
>
>
>
> When I Increase the kafkaconfig.fetchSizeBytes value to 150*1024*1024,I
> get desired output from kafka-spout but after sometime kafka spout is
> throwing the exception and my spout is failing.What should I do to handle
> this.Do I need to add any lines in storm.yaml to handle this exception.
>
> java.lang.OutOfMemoryError: Java heap space
> at java.util.Arrays.copyOf(Arrays.java:3332)
> at
> java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:137)
> at
> java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:121)
> at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:622)
> at java.lang.StringBuilder.append(StringBuilder.java:202)
> at java.util.AbstractCollection.toString(AbstractCollection.java:464)
> at clojure.core$str.invoke(core.clj:513)
> at clojure.core$str$fn__3896.invoke(core.clj:517)
> at clojure.core$str.doInvoke(core.clj:519)
> at clojure.lang.RestFn.invoke(RestFn.java:516)
> at backtype.storm.daemon.task$mk_tasks_fn$fn__6307.invoke(task.clj:152)
> at
> backtype.storm.daemon.executor$fn__6579$fn__6594$send_spout_msg__6612.invoke(executor.clj:482)
> at
> backtype.storm.daemon.executor$fn__6579$fn$reify__6621.emit(executor.clj:526)
> at
> backtype.storm.spout.SpoutOutputCollector.emit(SpoutOutputCollector.java:49)
> at
> backtype.storm.spout.SpoutOutputCollector.emit(SpoutOutputCollector.java:63)
> at storm.kafka.PartitionManager.next(PartitionManager.java:141)
> at storm.kafka.KafkaSpout.nextTuple(KafkaSpout.java:141)
> at
> backtype.storm.daemon.executor$fn__6579$fn__6594$fn__6623.invoke(executor.clj:565)
> at backtype.storm.util$async_loop$fn__459.invoke(util.clj:463)
> at clojure.lang.AFn.run(AFn.java:24)
> at java.lang.Thread.run(Thread.java:745)
>
> Need suggestions
>
>
>
>
> *Best regards,*
> *K.Sai Dilip Reddy.*
>
