HeartSaVioR commented on a change in pull request #24465:
[SPARK-27575][CORE][YARN] Yarn file-related confs should merge new value with
existing value
URL: https://github.com/apache/spark/pull/24465#discussion_r279099255
##########
File path: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala
##########
@@ -217,12 +233,16 @@ private[spark] class SparkSubmit extends Logging {
*/
private[deploy] def prepareSubmitEnvironment(
args: SparkSubmitArguments,
- conf: Option[HadoopConfiguration] = None)
+ conf: Option[HadoopConfiguration] = None,
+ sparkConfOpt: Option[SparkConf] = None)
Review comment:
There're couple of places which configuration is being set, and the place I
hit the issue was configuration file, so I was trying to make sure it is same.
But yeah it would be effectively testing the same (since `--conf` will be
handled before) so no big deal. Will roll back the change and use `--conf`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
