Lianhui Wang created SPARK-11252: ------------------------------------ Summary: ExternalShuffleClient should release connection after it had completed to fetch blocks from yarn's NameManager Key: SPARK-11252 URL: https://issues.apache.org/jira/browse/SPARK-11252 Project: Spark Issue Type: Bug Reporter: Lianhui Wang
ExternalShuffleClient of executors reserve its connection with yarn's NodeManager until application has been completed. so it will make NodeManager has many socket connections. in order to reduce network pressure of NodeManager's shuffleService, after registerWithShuffleServer or fetchBlocks have been completed in ExternalShuffleClient, connection for shuffleService needs to be closed. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org