advancedxy commented on code in PR #41201:
URL: https://github.com/apache/spark/pull/41201#discussion_r1211709330
##########
resource-managers/kubernetes/docker/src/main/dockerfiles/spark/entrypoint.sh:
##########
@@ -75,6 +75,9 @@ elif ! [ -z ${SPARK_HOME+x} ]; then
SPARK_CLASSPATH="$SPARK_HOME/conf:$SPARK_CLASSPATH";
fi
+# SPARK-43540: add current working directory into executor classpath
+SPARK_CLASSPATH="$PWD:$SPARK_CLASSPATH"
Review Comment:
> Sorry, but why this is prepended? For this specific part, I'm strong
negative because this could have a side-effect like
`spark.executor.userClassPathFirst=true`. As we know, we don't recommend
`userClassPathFirst` at all.
Hi @dongjoon-hyun, just to be clear, are you against for putting the `PWD`
into executor's class path or for the `PWD` being the first in the class path?
In my opinion, to align with spark on yarn's behavior, `PWD` should be put
in both driver and executor's class path. But I'm ok for it to be last or
anywhere in the class path.
By the way, this pr put `PWD(.)` in the first of class path for driver, if
you have concern about `PWD` being first, the driver may have the same issue
here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
