LingZhou created SPARK-11617:
--------------------------------
Summary: MEMORY LEAK: ByteBuf.release() was not called before it's
garbage-collected
Key: SPARK-11617
URL: https://issues.apache.org/jira/browse/SPARK-11617
Project: Spark
Issue Type: Bug
Components: Spark Core, YARN
Affects Versions: 1.6.0
Reporter: LingZhou
The problem may be related to
[SPARK-11235][NETWORK] Add ability to stream data using network lib.
while running on yarn-client mode, there are error messages:
15/11/09 10:23:55 ERROR util.ResourceLeakDetector: LEAK: ByteBuf.release() was
not called before it's garbage-collected. Enable advanced leak reporting to
find out where the leak occurred. To enable advanced leak reporting, specify
the JVM option '-Dio.netty.leakDetectionLevel=advanced' or call
ResourceLeakDetector.setLevel() See
http://netty.io/wiki/reference-counted-objects.html for more information.
and then it will cause
cluster.YarnSchedulerBackend$YarnSchedulerEndpoint: Container killed by YARN
for exceeding memory limits. 9.0 GB of 9 GB physical memory used. Consider
boosting spark.yarn.executor.memoryOverhead.
and WARN scheduler.TaskSetManager: Lost task 105.0 in stage 1.0 (TID 2616,
gsr489): java.lang.IndexOutOfBoundsException: index: 130828, length: 16833
(expected: range(0, 524288)).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
