attilapiros commented on pull request #29090: URL: https://github.com/apache/spark/pull/29090#issuecomment-658744201
@dongjoon-hyun Sure it can happen. I have updated the PR description see the section `Does this PR introduce any user-facing change?` listing the possible affected use cases. As I see k8s is not affected as `SparkContext` sets the `SPARK_EXECUTOR_MEMORY` environment variable with the correct suffix: https://github.com/apache/spark/blob/2ddbe0c9bf3fd3b0ea2933ddddf83324fc7566b9/core/src/main/scala/org/apache/spark/SparkContext.scala#L518 As an alternative solution we can only update our documentation emphasizing the importance to specify the suffix when a memory config is set. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
