Github user jiangxb1987 commented on the issue:
https://github.com/apache/spark/pull/20138
This PR introduces an more thoughtful event logs cleanup method, if users
have `spark.history.fs.cleaner.enabled` set to true, their existing invalid
event log files(empty/no valid applicationId) that have past the max log age
can now be deleted. It should be safe to do so because users should not have
files other than event log files in the directory. Can we merge this to
master/2.3? @sameeragarwal
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
