[
https://issues.apache.org/jira/browse/YARN-5325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15563165#comment-15563165
]
Carlo Curino commented on YARN-5325:
------------------------------------
Hi [~subru], the uploaded .09 version of the patch address most of your asks
above plus the discussion we had on having a fallback to "homesubcluster" in
case the resolver cannot locate the subcluster owning a node/rack. We also log
whether this happens. In case the homeSubCluster is not active, we do similarly
to what yarn would do and silently ignore the ResourceRequest (please confirm
this is ok for you).
A few minor things were not changed:
I fixed on AtomicLong that was redundandt, but the one in the HashMap are
useful to avoid recreating objects at every update (or lengthy code).
TreeMap is simply to have sorted output when printed (useful for debugging and
with <20 sub-clusters and no data-structure churn this should never be a per
issue).
The system.outs in tests are useful for debugging the tests. I would leave them
there if you are ok with it.
Exceptions in tests were not ignored, they had to be handled by the code later
on.
Regarding multi invocation, the only case for which it makes sense is for
"headroom" (as this is the only somewhat stateful side of the policy). I added
a second invocation in the corresponding test, to validate.
Regarding {{FederationPoliciesTestUtil::createResourceRequest}} there are few
objects to create, and this would make the code in tests quite redundant I
think.
Re thread safeness, the switch to re-using AllocationBookkeeper I think made
the code more fragile, so we should definitely double-check whether the
intended use
is single-threaded or protect the splitResourceRequest() with a lock.
> Stateless ARMRMProxy policies implementation
> --------------------------------------------
>
> Key: YARN-5325
> URL: https://issues.apache.org/jira/browse/YARN-5325
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: nodemanager, resourcemanager
> Affects Versions: YARN-2915
> Reporter: Carlo Curino
> Assignee: Carlo Curino
> Attachments: YARN-5325-YARN-2915.05.patch,
> YARN-5325-YARN-2915.06.patch, YARN-5325-YARN-2915.07.patch,
> YARN-5325-YARN-2915.08.patch, YARN-5325-YARN-2915.09.patch,
> YARN-5325.01.patch, YARN-5325.02.patch, YARN-5325.03.patch, YARN-5325.04.patch
>
>
> This JIRA tracks policies in the AMRMProxy that decide how to forward
> ResourceRequests, without maintaining substantial state across decissions
> (e.g., broadcast).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
