Hi, please have a look at log snippet: 16/05/18 03:27:16 INFO spark.MapOutputTrackerWorker: Doing the fetch; tracker endpoint = NettyRpcEndpointRef(spark://mapoutputtrac...@xxx.xxx.xxx.xxx:38128) 16/05/18 03:27:16 INFO spark.MapOutputTrackerWorker: Got the output locations 16/05/18 03:27:16 INFO storage.ShuffleBlockFetcherIterator: Getting 30 non-empty blocks out of 30 blocks 16/05/18 03:27:16 INFO storage.ShuffleBlockFetcherIterator: Started 30 remote fetches in 3 ms 16/05/18 03:27:16 INFO spark.MapOutputTrackerWorker: Don't have map outputs for shuffle 1, fetching them 16/05/18 03:27:16 INFO spark.MapOutputTrackerWorker: Doing the fetch; tracker endpoint = NettyRpcEndpointRef(spark://mapoutputtrac...@xxx.xxx.xxx.xxx:38128) 16/05/18 03:27:16 INFO spark.MapOutputTrackerWorker: Got the output locations 16/05/18 03:27:16 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1500 blocks 16/05/18 03:27:16 INFO storage.ShuffleBlockFetcherIterator: Started 1 remote fetches in 1 ms 16/05/18 03:27:17 ERROR executor.Executor: Managed memory leak detected; size = 6685476 bytes, TID = 3405 16/05/18 03:27:17 ERROR executor.Executor: Exception in task 285.0 in stage 6.0 (TID 3405)
Is it related to https://issues.apache.org/jira/browse/SPARK-11293 Is there any recommended workaround?
