ewoodbury commented on PR #3879: URL: https://github.com/apache/celeborn/pull/3879#issuecomment-6282229467
Thanks @zaynt4606! On the destroy, that's a good point, it makes sense to track these failure better. Looks like the update to cover this would be slim (<100 new lines) so I'll plan to do that: updating StorageManager.cleanFile so a failed delete is not reported as success. It'll keep the file info and writer, and throw so `DestroyWorkerSlots` replies PARTIAL_SUCCESS and the client retries. If the retries also fail, the file stays tracked and is removed at shuffle expiry. Will add a failure injection test as well, covering the updated path as well, makes sense -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
