[
https://issues.apache.org/jira/browse/YARN-7972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16384418#comment-16384418
]
Weiwei Yang edited comment on YARN-7972 at 3/3/18 1:48 AM:
-----------------------------------------------------------
Hi [~asuresh]
Thanks for your comments, I introduced the Evaluable interface is to separate
1) parsing phase and 2) eval phase, before a namespace can be consumed by the
allocation tags manager. This is because during phase 1, we are not able to get
the exact scope of a namespace, as the parser has no concept of what is the
current application, other applications or all applications etc. So this will
be handled in phase 2. I think this makes code more readable.
{quote}
then we must ensure evaluate is always required to be called
{quote}
Makes sense. To achieve this, I can try to add a new class to represent the
eval target, e.g {{TargetApplications}}
{code}
TargetApplications ta = new TargetApplications(currentAppId, allAppIds);
// this way we will get rid of if-else code like you pointed out, and move the
logic to the eval method itself
namespace.evaluate(ta);
{code}
If this looks good to you, I will submit a new patch with this change.
{quote}
I assume later when we have NOT_SELF and multiple app scopes, then the size of
set would be all other apps and each specific app respectively. right ?
{quote}
Correct. The idea is the eval phase will make sure a namespace is interpreted
to a specific scope, AKA a set of app IDs. NOT_SELF, ALL, APP-LABEL will all
need to implement its own eval method.
{quote}
Wouldn't creating that set at parse time be very in-efficient ?
{quote}
I am not sure if I get your point correctly, we are not creating a new set
every time, we only need to pass current app ID and all app IDs (from
allocation tags manager) to a namespace so it can be evaluated to a specific
scope according to different namespace types. Would that be a concern?
Thanks
was (Author: cheersyang):
Hi [~asuresh]
Thanks for your comments, I introduced the Evaluable interface is to separate
1) parsing phase and 2) eval phase, before a namespace can be consumed by the
allocation tags manager. This is because during phase 1, we are not able to get
the exact scope of a namespace during parsing phase, as the parser has no
concept of what is the current application, other applications or all
applications etc. So this will be handled in phase 2. I think this makes code
more readable.
{quote}
then we must ensure evaluate is always required to be called
{quote}
Makes sense. To achieve this, I can try to add a new class to represent the
eval target, e.g {{TargetApplications}}
{code}
TargetApplications ta = new TargetApplications(currentAppId, allAppIds);
// this way we will get rid of if-else code like you pointed out, and move the
logic to the eval method itself
namespace.evaluate(ta);
{code}
If this looks good to you, I will submit a new patch with this change.
{quote}
I assume later when we have NOT_SELF and multiple app scopes, then the size of
set would be all other apps and each specific app respectively. right ?
{quote}
Correct. The idea is the eval phase will make sure a namespace is interpreted
to a specific scope, AKA a set of app IDs. NOT_SELF, ALL, APP-LABEL will all
need to implement its own eval method.
{quote}
Wouldn't creating that set at parse time be very in-efficient ?
{quote}
I am not sure if I get your point correctly, we are not creating a new set
every time, we only need to pass current app ID and all app IDs (from
allocation tags manager) to a namespace so it can be evaluated to a specific
scope according to different namespace types. Would that be a concern?
Thanks
> Support inter-app placement constraints for allocation tags by application ID
> -----------------------------------------------------------------------------
>
> Key: YARN-7972
> URL: https://issues.apache.org/jira/browse/YARN-7972
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Weiwei Yang
> Assignee: Weiwei Yang
> Priority: Major
> Attachments: YARN-7972.001.patch, YARN-7972.002.patch,
> YARN-7972.003.patch, YARN-7972.004.patch, YARN-7972.005.patch,
> YARN-7972.006.patch
>
>
> Per discussion in [this
> comment|https://issues.apache.org/jira/browse/YARN-6599focusedCommentId=16319662&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16319662]
> in YARN-6599, we need to support inter-app PC for allocation tags.
> This will help to do better placement when dealing with potential competing
> resource applications, e.g don't place two tensorflow workers from two
> different applications on one same node.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
