DB Tsai created SPARK-31582:
-------------------------------
Summary: Be able to not populate Hadoop classpath
Key: SPARK-31582
URL: https://issues.apache.org/jira/browse/SPARK-31582
Project: Spark
Issue Type: New Feature
Components: YARN
Affects Versions: 2.4.5
Reporter: DB Tsai
Spark Yarn client will populate hadoop classpath from
`yarn.application.classpath` and ``mapreduce.application.classpath`. However,
for Spark with embedded hadoop build, it will result jar conflicts because
spark distribution can contain different version of hadoop jars.
We are adding a new Yarn configuration to not populate hadoop classpath from
`yarn.application.classpath` and ``mapreduce.application.classpath`.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
