Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/2350#discussion_r17430322
--- Diff:
yarn/common/src/main/scala/org/apache/spark/deploy/yarn/YarnSparkHadoopUtil.scala
---
@@ -142,14 +136,13 @@ object YarnSparkHadoopUtil {
m.appendReplacement(sb, Matcher.quoteReplacement(replace))
}
m.appendTail(sb)
- addToEnvironment(env, parts(0), sb.toString(), classPathSeparator)
--- End diff --
It's just very strange that we're appending the variable and delimiting it
with the path separator for everything. For instance, if I set something
unrelated that has already been set:
```
export SPARK_YARN_USER_ENV=ENABLE_SOME_FEATURE=true
```
In the containers `ENABLE_SOME_FEATURE` might instead have the value
`false:true`. In general this is not the behavior I would expect for setting
environment variables in other settings.
I can change it back if you think it's important for backward compatibility.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
