HeartSaVioR commented on a change in pull request #19616: [SPARK-22404][YARN]
Provide an option to use unmanaged AM in yarn-client mode
URL: https://github.com/apache/spark/pull/19616#discussion_r251184734
##########
File path:
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ExecutorRunnable.scala
##########
@@ -243,7 +243,7 @@ private[yarn] class ExecutorRunnable(
// Add log urls
container.foreach { c =>
- sys.env.get("SPARK_USER").foreach { user =>
+ sys.env.filterKeys(_.endsWith("USER")).foreach { user =>
Review comment:
@devaraj-kavali @vanzin
While resolving merge conflict in #23260, I found two observations here:
1. What output we expect when `env` have more than one matching keys? This
looks like always leveraging the last key which is indeterministic if there're
more than one keys being matched.
2. Here the `user` is not a `value` but a `(key, value)` here, so we need to
use either key or value (I guess we would like to pick `value`).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
