garyli1019 commented on issue #768: No Space Left On Device for upsert URL: https://github.com/apache/incubator-hudi/issues/768#issuecomment-512514758 https://issues.apache.org/jira/browse/HUDI-171 @vinothchandar In my cluster set up, all the spark shuffle services are not using `/tmp`, so I think those files are left behind by hudi. Example of a file left in `/tmp`: `-rw-r----- 1 u_ops 168M Jul 14 18:10 d7b2a7a3-5706-4ffd-90cb-70c6650ef1e4` I think we can find a way to predict the file size before actually writing to tmp. It will be difficult to go back to the worker node to delete those files after the job failed.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
