koertkuipers commented on issue #24465: [SPARK-27575][CORE][YARN] Yarn file-related confs should merge new value with existing value URL: https://github.com/apache/spark/pull/24465#issuecomment-487688376 this is nice improvement. however doesnt the same apply to: spark.driver.extraClassPath spark.driver.extraLibraryPath spark.driver.extraJavaOptions spark.executor.extraClassPath spark.executor.extraLibraryPath spark.executor.extraJavaOptions and probably others as well? i have often ran into many bugs because someone needed to set an extra java option and didnt realize they were overwriting existing settings (for example in spark-defaults.conf). same with classpath and library path.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
