[ 
https://issues.apache.org/jira/browse/YARN-8394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16671119#comment-16671119
 ] 

Weiwei Yang commented on YARN-8394:
-----------------------------------

Hi [~yufeigu]

Apologies I missed your last comment.

Are you suggesting when "yarn.scheduler.capacity.node-locality-delay" is set to 
"-1", then we should AUTO disable "rack-locality-additional-delay" too?  I 
think that makes sense.

We need a Jira to track this, revisit the locality code under the context of 
cloud environment. Let me open one to track.

Thanks!

> Improve data locality documentation for Capacity Scheduler
> ----------------------------------------------------------
>
>                 Key: YARN-8394
>                 URL: https://issues.apache.org/jira/browse/YARN-8394
>             Project: Hadoop YARN
>          Issue Type: Improvement
>            Reporter: Weiwei Yang
>            Assignee: Weiwei Yang
>            Priority: Major
>             Fix For: 3.2.0, 3.1.1, 3.0.4
>
>         Attachments: YARN-8394.001.patch, YARN-8394.002.patch
>
>
> YARN-6344 introduces a new parameter 
> {{yarn.scheduler.capacity.rack-locality-additional-delay}} in 
> capacity-scheduler.xml, we need to add some documentation in 
> {{CapacityScheduler.md}} accordingly.
> Moreover, we are seeing more and more clusters are separating storage and 
> computation where file system is always remote, in such cases we need to 
> introduce how to compromise data locality in CS otherwise MR jobs are 
> suffering.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

Reply via email to