tgravescs commented on pull request #30710: URL: https://github.com/apache/spark/pull/30710#issuecomment-756428929
I commented on the jira, this is probably a dup of https://issues.apache.org/jira/browse/SPARK-29910, you can see there was a PR a little discussion there. While I agree it might not be optimal and configuring per stage and such might be better, that is much harder and it seems like this would be good first start and its just changing it to be configurable. you will want to change the config to be in core/src/main/scala/org/apache/spark/internal/config/package.scala rather then a hardcoded string. I assume you have seen some direct use cases this applies? ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
