Please increase the number of partitions. Cheers
On Wed, May 18, 2016 at 4:17 AM, Serega Sheypak <serega.shey...@gmail.com> wrote: > Hi, please have a look at log snippet: > 16/05/18 03:27:16 INFO spark.MapOutputTrackerWorker: Doing the fetch; > tracker endpoint = > NettyRpcEndpointRef(spark://mapoutputtrac...@xxx.xxx.xxx.xxx:38128) > 16/05/18 03:27:16 INFO spark.MapOutputTrackerWorker: Got the output > locations > 16/05/18 03:27:16 INFO storage.ShuffleBlockFetcherIterator: Getting 30 > non-empty blocks out of 30 blocks > 16/05/18 03:27:16 INFO storage.ShuffleBlockFetcherIterator: Started 30 > remote fetches in 3 ms > 16/05/18 03:27:16 INFO spark.MapOutputTrackerWorker: Don't have map > outputs for shuffle 1, fetching them > 16/05/18 03:27:16 INFO spark.MapOutputTrackerWorker: Doing the fetch; > tracker endpoint = > NettyRpcEndpointRef(spark://mapoutputtrac...@xxx.xxx.xxx.xxx:38128) > 16/05/18 03:27:16 INFO spark.MapOutputTrackerWorker: Got the output > locations > 16/05/18 03:27:16 INFO storage.ShuffleBlockFetcherIterator: Getting 1 > non-empty blocks out of 1500 blocks > 16/05/18 03:27:16 INFO storage.ShuffleBlockFetcherIterator: Started 1 > remote fetches in 1 ms > 16/05/18 03:27:17 ERROR executor.Executor: Managed memory leak detected; > size = 6685476 bytes, TID = 3405 > 16/05/18 03:27:17 ERROR executor.Executor: Exception in task 285.0 in > stage 6.0 (TID 3405) > > Is it related to https://issues.apache.org/jira/browse/SPARK-11293 > > Is there any recommended workaround? >
