srowen commented on code in PR #42313:
URL: https://github.com/apache/spark/pull/42313#discussion_r1284387913
##########
core/src/main/scala/org/apache/spark/SparkConf.scala:
##########
@@ -518,16 +516,19 @@ class SparkConf(loadDefaults: Boolean) extends Cloneable
with Logging with Seria
}
// Validate spark.executor.extraJavaOptions
- getOption(executorOptsKey).foreach { javaOpts =>
- if (javaOpts.contains("-Dspark")) {
- val msg = s"$executorOptsKey is not allowed to set Spark options (was
'$javaOpts'). " +
- "Set them directly on a SparkConf or in a properties file when using
./bin/spark-submit."
- throw new Exception(msg)
- }
- if (javaOpts.contains("-Xmx")) {
- val msg = s"$executorOptsKey is not allowed to specify max heap memory
settings " +
- s"(was '$javaOpts'). Use spark.executor.memory instead."
- throw new Exception(msg)
+ Seq(EXECUTOR_JAVA_OPTIONS.key,
"spark.executor.defaultJavaOptions").foreach { executorOptsKey =>
Review Comment:
Is there not a constant already for `spark.executor.defaultJavaOptions`? not
a big deal
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
