[
https://issues.apache.org/jira/browse/YARN-4002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15387088#comment-15387088
]
Guoyao Xu commented on YARN-4002:
---------------------------------
I think that might not be a big problem. Because hostsReader.refresh()
operation does not happen during each heartbeat, and actually
this operation is not related to isValid() operation. And the frequency of
refresh() operation is much less than isValid() operation (nodes number times
of call for each heartbeat interval--- default 1 second).
Then then the synchronized lock of refresh() may not be a problem of isValid().
> make ResourceTrackerService.nodeHeartbeat more concurrent
> ---------------------------------------------------------
>
> Key: YARN-4002
> URL: https://issues.apache.org/jira/browse/YARN-4002
> Project: Hadoop YARN
> Issue Type: Improvement
> Reporter: Hong Zhiguo
> Assignee: Hong Zhiguo
> Priority: Critical
> Fix For: 2.8.0
>
> Attachments: 0001-YARN-4002.patch, YARN-4002-lockless-read.patch,
> YARN-4002-rwlock-v2.patch, YARN-4002-rwlock-v2.patch,
> YARN-4002-rwlock-v3-rebase.patch, YARN-4002-rwlock-v3.patch,
> YARN-4002-rwlock-v4.patch, YARN-4002-rwlock-v5.patch,
> YARN-4002-rwlock-v6.patch, YARN-4002-rwlock.patch, YARN-4002-v0.patch
>
>
> We have multiple RPC threads to handle NodeHeartbeatRequest from NMs. By
> design the method ResourceTrackerService.nodeHeartbeat should be concurrent
> enough to scale for large clusters.
> But we have a "BIG" lock in NodesListManager.isValidNode which I think it's
> unnecessary.
> First, the fields "includes" and "excludes" of HostsFileReader are only
> updated on "refresh nodes". All RPC threads handling node heartbeats are
> only readers. So RWLock could be used to alow concurrent access by RPC
> threads.
> Second, since he fields "includes" and "excludes" of HostsFileReader are
> always updated by "reference assignment", which is atomic in Java, the reader
> side lock could just be skipped.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
