youngseaz created SPARK-50142:
---------------------------------
Summary: Unusual parameter behavior
Key: SPARK-50142
URL: https://issues.apache.org/jira/browse/SPARK-50142
Project: Spark
Issue Type: Improvement
Components: Spark Submit
Affects Versions: 3.5.3
Reporter: youngseaz
dear, The parameters of spark-mission submitted by spark-submit should not be
parsed into commands but are parsed into commands，such as
*spark.executor.extraJavaOptions，* *spark.executor.defaultJavaOptions,*
*spark.executor.extraJavaOptions,* *spark.driver.defaultJavaOptions.* These
parameters are used in the direct splice command and the command be executed in
yarn mode (bash -c "xxxx" ).
*spark.executor.extraJavaOptions is used in the direct splice command:*
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ExecutorRunnable.scala
{quote}private def prepareCommand(): List[String] = {
// Extra options for the JVM
val javaOpts = ListBuffer[String]()
// Set the JVM memory
val executorMemoryString = executorMemory + "m"
javaOpts += "-Xmx" + executorMemoryString
// Set extra Java options for the executor, if defined
sparkConf.get(EXECUTOR_JAVA_OPTIONS).foreach { opts =>
val subsOpt = Utils.substituteAppNExecIds(opts, appId, executorId)
javaOpts ++=
Utils.splitCommandString(subsOpt).map(YarnSparkHadoopUtil.escapeForShell)
}
{quote}
>From a security perspective, data and code (commands) should be handled
>separately. These parameters can be executed as codes (commands). A sugesstion
>for this issue that is recommended to verify these parameters to enhance
>security.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
