[
https://issues.apache.org/jira/browse/YARN-8428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eric Yang resolved YARN-8428.
-----------------------------
Resolution: Won't Fix
YARN ServiceClient is using a single connection to connect to ZooKeeper for
deleting DNS entries. The number of ZooKeeper connection required by the
framework depends on number of YARN service applications running on the
cluster. This is a design trade off for using ZooKeeper as storage for
coordinating distributed applications. The only solution is to increase number
of ZooKeeper concurrent connections, i.e. maxClientCnxns=2000 to increase
number of YARN services can run on the cluster. Close this as working as
designed.
> YARN service has ZooKeeper connection leaks
> -------------------------------------------
>
> Key: YARN-8428
> URL: https://issues.apache.org/jira/browse/YARN-8428
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn-native-services
> Affects Versions: 3.1.0
> Reporter: Eric Yang
> Assignee: Eric Yang
> Priority: Critical
>
> There are two leaks discovered during stress test. First, YARN service
> application master uses RegistryDNS to create DNS record for YARN service.
> Data is first persisted on ZooKeeper, and RegistryDNS will lookup data in
> ZooKeeper to create DNS entries. YARN service AM does not close ZooKeeper
> connection after the record has been created. Second, when deleting DNS
> record from ZooKeeper, ZooKeeper connection is left open.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
