Vinod Kumar Vavilapalli commented on YARN-3626:

bq. I thought this implemented "per app" behavior, because it could vary 
between MR app submission instances. It would not be a requirement to put 
the client and server share configs.
YARN doesn't have a notion of app-configs, it doesn't know app's config files 
etc. So, the app cannot set a config property that it expects the server to 

No idea, how your original patch apparently worked. May be we are missing 

[~cwelch], what I was proposing was something in the lines of (a) user sets MR 
user-classpath-before config (b) MR converts that into a special env for YARN 
and (c) YARN looks at the ENV to figure out how to the order the classpath.

Overall, it is terrible that we are talking classpaths in YARN, but that's for 
another JIRA.

> On Windows localized resources are not moved to the front of the classpath 
> when they should be
> ----------------------------------------------------------------------------------------------
>                 Key: YARN-3626
>                 URL: https://issues.apache.org/jira/browse/YARN-3626
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: yarn
>         Environment: Windows
>            Reporter: Craig Welch
>            Assignee: Craig Welch
>             Fix For: 2.7.1
>         Attachments: YARN-3626.0.patch, YARN-3626.4.patch, YARN-3626.6.patch, 
> YARN-3626.9.patch
> In response to the mapreduce.job.user.classpath.first setting the classpath 
> is ordered differently so that localized resources will appear before system 
> classpath resources when tasks execute.  On Windows this does not work 
> because the localized resources are not linked into their final location when 
> the classpath jar is created.  To compensate for that localized jar resources 
> are added directly to the classpath generated for the jar rather than being 
> discovered from the localized directories.  Unfortunately, they are always 
> appended to the classpath, and so are never preferred over system resources.

This message was sent by Atlassian JIRA

Reply via email to