ScrapCodes opened a new pull request #26193: [WIP][SPARK-25065][k8s] Allow setting up correct logging configuration on driver and executor. URL: https://github.com/apache/spark/pull/26193 This is work in progress, because I wanted some early feedback as to overall approach taken. At the moment with current code, this mounting feature does not work in client mode of submission. The reason is, in client mode kubernetes does not create any resources. My question is, what is the correct way to create a config map, while in the client mode. If it is created in a place like ExecutorPodAllocator, then since it is run repeatedly, then we would not want the config map to be created again and again on each executor pod allocation. So, what would be correct place to do it. ### What changes were proposed in this pull request? A new feature to propagate correct logging configuration file either by picking up from user specified logger configuration file on the classpath or through user specified k8s config map. ### Why are the changes needed? Please see referenced JIRA. ### Does this PR introduce any user-facing change? No ### How was this patch tested? Manually tested.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
