ScrapCodes commented on pull request #30472: URL: https://github.com/apache/spark/pull/30472#issuecomment-738634123
> `truncateToSize` seems to ignore users' files without proper warnings. > Instead of config data loss, why don't we make Spark job fail in this case? Yeah, it makes sense to fail with a good explanation. One of my considerations were, by failing nicely we are changing the existing behaviour (as in 3.0.x) i.e. some invalid or large configuration file in SPARK_CONF_DIR did not fail the job run. After this change, those who upgrade, will have to make changes to their config directory as well. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
