LuciferYang commented on code in PR #58343: URL: https://github.com/apache/spark/pull/58343#discussion_r4022015356
########## docs/core-migration-guide.md: ########## @@ -42,6 +42,8 @@ license: | - Since Spark 4.3, a positive `spark.executor.pyspark.memory` allocation that is too small to give each concurrent task slot at least 1 MiB fails the Python task with an error instead of silently running the workers without any memory limit. Setting `spark.executor.pyspark.memory=0` still disables the limit. To restore a working memory limit, increase `spark.executor.pyspark.memory` or reduce the executor's concurrent task capacity. +- Since Spark 4.3, an executor pod template that names a service account in `serviceAccountName` keeps it: Spark applies `spark.kubernetes.authenticate.executor.serviceAccountName`, or the driver's account as a fallback, only when the template names no account in either `serviceAccount` or `serviceAccountName`. Earlier versions decided by reading the deprecated `serviceAccount` field alone, so a template that named the account in `serviceAccountName` had it overwritten. Spark logs a warning when `spark.kubernetes.authenticate.executor.serviceAccountName` named an account the template displaced. Review Comment: Ok, I'll change it to Spark 4.4. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
