Chris Nauroth commented on YARN-3685:

bq. Which ones are these?

I was thinking of stuff like {{yarn.application.classpath}}, where values are 
defined in terms of things like the {{HADOOP_YARN_HOME}} and 
{{HADOOP_COMMON_HOME}} environment variables, and those values might not match 
the file system layout at the client side.

bq. Not clear this is true or not. Have to see the final solution/patch to 
realistically reason about this.

Let's say hypothetically a change for this goes into 2.8.0.  I was thinking 
that would make it impossible for a 2.7.0 client to work correctly with a 2.8.0 
NodeManager, because that client wouldn't take care of classpath bundling and 
instead expect the NodeManager to do it.

Brainstorming a bit, maybe we can figure out a way for a 2.8.0 NodeManager to 
detect if the client hasn't already taken care of classpath bundling, and if 
not, stick to the current logic.  Backwards-compatibility logic like this would 
go into branch-2, but could be dropped from trunk.

> NodeManager unnecessarily knows about classpath-jars due to Windows 
> limitations
> -------------------------------------------------------------------------------
>                 Key: YARN-3685
>                 URL: https://issues.apache.org/jira/browse/YARN-3685
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: nodemanager
>            Reporter: Vinod Kumar Vavilapalli
> Found this while looking at cleaning up ContainerExecutor via YARN-3648, 
> making it a sub-task.
> YARN *should not* know about classpaths. Our original design modeled around 
> this. But when we added windows suppport, due to classpath issues, we ended 
> up breaking this abstraction via YARN-316. We should clean this up.

This message was sent by Atlassian JIRA

Reply via email to