shanyu commented on issue #27781: [SPARK-31028][CORE] Add "-XX:ActiveProcessorCount" to Spark driver and executor in Yarn mode URL: https://github.com/apache/spark/pull/27781#issuecomment-596127646 Thanks @dongjoon-hyun for reviewing this PR. spark.driver.extraJavaOptions and spark.executor.extraJavaOptions could be a workaround. However, it should not be a long term solution, right? Customer already specified spark.driver.cores and spark.executor.cores and we should honor it. I don't know which spark version target java 8u191 and above, and we should put the fix in for such spark version in the future then.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
