[ 
https://issues.apache.org/jira/browse/YARN-7972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16384418#comment-16384418
 ] 

Weiwei Yang edited comment on YARN-7972 at 3/3/18 1:48 AM:
-----------------------------------------------------------

Hi [~asuresh]

Thanks for your comments, I introduced the Evaluable interface is to separate 
1) parsing phase and 2) eval phase, before a namespace can be consumed by the 
allocation tags manager. This is because during phase 1, we are not able to get 
the exact scope of a namespace, as the parser has no concept of what is the 
current application, other applications or all applications etc. So this will 
be handled in phase 2. I think this makes code more readable.

{quote}
then we must ensure evaluate is always required to be called
{quote}
Makes sense. To achieve this, I can try to add a new class to represent the 
eval target, e.g {{TargetApplications}}
{code}
TargetApplications ta = new TargetApplications(currentAppId, allAppIds);
// this way we will get rid of if-else code like you pointed out, and move the 
logic to the eval method itself
namespace.evaluate(ta);
{code}
If this looks good to you, I will submit a new patch with this change.

{quote}
I assume later when we have NOT_SELF and multiple app scopes, then the size of 
set would be all other apps and each specific app respectively. right ?
{quote}
Correct. The idea is the eval phase will make sure a namespace is interpreted 
to a specific scope, AKA a set of app IDs. NOT_SELF, ALL, APP-LABEL will all 
need to implement its own eval method.

{quote}
Wouldn't creating that set at parse time be very in-efficient ?
{quote}
I am not sure if I get your point correctly, we are not creating a new set 
every time, we only need to pass current app ID and all app IDs (from 
allocation tags manager) to a namespace so it can be evaluated to a specific 
scope according to different namespace types. Would that be a concern?

Thanks

 


was (Author: cheersyang):
Hi [~asuresh]

Thanks for your comments, I introduced the Evaluable interface is to separate 
1) parsing phase and 2) eval phase, before a namespace can be consumed by the 
allocation tags manager. This is because during phase 1, we are not able to get 
the exact scope of a namespace during parsing phase, as the parser has no 
concept of what is the current application, other applications or all 
applications etc. So this will be handled in phase 2. I think this makes code 
more readable.

{quote}
then we must ensure evaluate is always required to be called
{quote}
Makes sense. To achieve this, I can try to add a new class to represent the 
eval target, e.g {{TargetApplications}}
{code}
TargetApplications ta = new TargetApplications(currentAppId, allAppIds);
// this way we will get rid of if-else code like you pointed out, and move the 
logic to the eval method itself
namespace.evaluate(ta);
{code}
If this looks good to you, I will submit a new patch with this change.

{quote}
I assume later when we have NOT_SELF and multiple app scopes, then the size of 
set would be all other apps and each specific app respectively. right ?
{quote}
Correct. The idea is the eval phase will make sure a namespace is interpreted 
to a specific scope, AKA a set of app IDs. NOT_SELF, ALL, APP-LABEL will all 
need to implement its own eval method.

{quote}
Wouldn't creating that set at parse time be very in-efficient ?
{quote}
I am not sure if I get your point correctly, we are not creating a new set 
every time, we only need to pass current app ID and all app IDs (from 
allocation tags manager) to a namespace so it can be evaluated to a specific 
scope according to different namespace types. Would that be a concern?

Thanks

 

> Support inter-app placement constraints for allocation tags by application ID
> -----------------------------------------------------------------------------
>
>                 Key: YARN-7972
>                 URL: https://issues.apache.org/jira/browse/YARN-7972
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Weiwei Yang
>            Assignee: Weiwei Yang
>            Priority: Major
>         Attachments: YARN-7972.001.patch, YARN-7972.002.patch, 
> YARN-7972.003.patch, YARN-7972.004.patch, YARN-7972.005.patch, 
> YARN-7972.006.patch
>
>
> Per discussion in [this 
> comment|https://issues.apache.org/jira/browse/YARN-6599focusedCommentId=16319662&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16319662]
>  in  YARN-6599, we need to support inter-app PC for allocation tags.
> This will help to do better placement when dealing with potential competing 
> resource applications, e.g don't place two tensorflow workers from two 
> different applications on one same node.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

Reply via email to