Ted-Jiang opened a new issue #1780: URL: https://github.com/apache/arrow-datafusion/issues/1780
**Is your feature request related to a problem or challenge? Please describe what you are trying to do.** Enable periodic cleanup of work_dir directories in ballista executor which introduce 3 args `executor_cleanup_enable` : Enable periodic cleanup of work_dir directories. `executor_cleanup_interval`: Controls the interval in seconds , which the worker cleans up old job dirs on the local machine. `executor_cleanup_ttl`: Number of seconds to retain job work_dir on each executor. This is a Time To Live and should depend on the amount of available disk space you have. **Describe the solution you'd like** Executor periodic spawn a task to clean work_dir, if all the files in `job_dir` not modified in `executor_cleanup_ttl` seconds, it will be deleted. **Describe alternatives you've considered** Scheduler send rpc call to delete files when job done. **Additional context** #1662 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
