[ https://issues.apache.org/jira/browse/YARN-2314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14177169#comment-14177169 ]
Wangda Tan commented on YARN-2314: ---------------------------------- [~jlowe], thanks for update, patch looks good to me, +1! [~rajesh.balamohan], thanks for your performance report based on this. 20-30ms is still a latency for interactive tasks cannot be totally ignored. At least, we have way to cache connections via configuration option in this patch. Wangda > ContainerManagementProtocolProxy can create thousands of threads for a large > cluster > ------------------------------------------------------------------------------------ > > Key: YARN-2314 > URL: https://issues.apache.org/jira/browse/YARN-2314 > Project: Hadoop YARN > Issue Type: Bug > Components: client > Affects Versions: 2.1.0-beta > Reporter: Jason Lowe > Assignee: Jason Lowe > Priority: Critical > Attachments: YARN-2314.patch, YARN-2314v2.patch, > disable-cm-proxy-cache.patch, nmproxycachefix.prototype.patch, > tez-yarn-2314.xlsx > > > ContainerManagementProtocolProxy has a cache of NM proxies, and the size of > this cache is configurable. However the cache can grow far beyond the > configured size when running on a large cluster and blow AM address/container > limits. More details in the first comment. -- This message was sent by Atlassian JIRA (v6.3.4#6332)