Chris Nauroth commented on YARN-3626:

I don't fully understand the objection to the former patch that had been 

bq. The new configuration added is supposed to be per app, but it is now a 
server side configuration.

There was a new YARN configuration property for triggering this behavior, but 
the MR application would toggle on that YARN property only if the MR job 
submission had {{MAPREDUCE_JOB_USER_CLASSPATH_FIRST}} on.  From {{MRApps}}:

    boolean userClassesTakesPrecedence = 
      conf.getBoolean(MRJobConfig.MAPREDUCE_JOB_USER_CLASSPATH_FIRST, false);

    if (userClassesTakesPrecedence) {

I thought this implemented "per app" behavior, because it could vary between MR 
app submission instances.  It would not be a requirement to put 
have the client and server share configs.

Is there a detail I'm missing?

> On Windows localized resources are not moved to the front of the classpath 
> when they should be
> ----------------------------------------------------------------------------------------------
>                 Key: YARN-3626
>                 URL: https://issues.apache.org/jira/browse/YARN-3626
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: yarn
>         Environment: Windows
>            Reporter: Craig Welch
>            Assignee: Craig Welch
>             Fix For: 2.7.1
>         Attachments: YARN-3626.0.patch, YARN-3626.4.patch, YARN-3626.6.patch, 
> YARN-3626.9.patch
> In response to the mapreduce.job.user.classpath.first setting the classpath 
> is ordered differently so that localized resources will appear before system 
> classpath resources when tasks execute.  On Windows this does not work 
> because the localized resources are not linked into their final location when 
> the classpath jar is created.  To compensate for that localized jar resources 
> are added directly to the classpath generated for the jar rather than being 
> discovered from the localized directories.  Unfortunately, they are always 
> appended to the classpath, and so are never preferred over system resources.

This message was sent by Atlassian JIRA

Reply via email to