[ 
https://issues.apache.org/jira/browse/YARN-3626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14546365#comment-14546365
 ] 

Chris Nauroth commented on YARN-3626:
-------------------------------------

I don't fully understand the objection to the former patch that had been 
committed.

bq. The new configuration added is supposed to be per app, but it is now a 
server side configuration.

There was a new YARN configuration property for triggering this behavior, but 
the MR application would toggle on that YARN property only if the MR job 
submission had {{MAPREDUCE_JOB_USER_CLASSPATH_FIRST}} on.  From {{MRApps}}:

{code}
    boolean userClassesTakesPrecedence = 
      conf.getBoolean(MRJobConfig.MAPREDUCE_JOB_USER_CLASSPATH_FIRST, false);

    if (userClassesTakesPrecedence) {
      conf.set(YarnConfiguration.YARN_APPLICATION_CLASSPATH_PREPEND_DISTCACHE,
        "true");
    }
{code}

I thought this implemented "per app" behavior, because it could vary between MR 
app submission instances.  It would not be a requirement to put 
{{YARN_APPLICATION_CLASSPATH_PREPEND_DISTCACHE}} into the server configs and 
have the client and server share configs.

Is there a detail I'm missing?

> On Windows localized resources are not moved to the front of the classpath 
> when they should be
> ----------------------------------------------------------------------------------------------
>
>                 Key: YARN-3626
>                 URL: https://issues.apache.org/jira/browse/YARN-3626
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: yarn
>         Environment: Windows
>            Reporter: Craig Welch
>            Assignee: Craig Welch
>             Fix For: 2.7.1
>
>         Attachments: YARN-3626.0.patch, YARN-3626.4.patch, YARN-3626.6.patch, 
> YARN-3626.9.patch
>
>
> In response to the mapreduce.job.user.classpath.first setting the classpath 
> is ordered differently so that localized resources will appear before system 
> classpath resources when tasks execute.  On Windows this does not work 
> because the localized resources are not linked into their final location when 
> the classpath jar is created.  To compensate for that localized jar resources 
> are added directly to the classpath generated for the jar rather than being 
> discovered from the localized directories.  Unfortunately, they are always 
> appended to the classpath, and so are never preferred over system resources.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Reply via email to