ewoodbury commented on PR #3879: URL: https://github.com/apache/celeborn/pull/3879#issuecomment-6304336749
Thanks @SteNicholas , good catch - Fixed so `DfsFlushTask.flush` now re-throws a stream-close failure instead of just logging it. If the write itself failed, that error is kept and the close error is added as suppressed. The flusher sets it on the notifier, so the writer's `close()` fails the commit, that copy is reported failed, and its replica is never released. Added a regression test with stream reuse disabled, where the append stream drops the data and fails on close. Also rebased onto main to pick up the CI fix; only new change is the last commit: https://github.com/apache/celeborn/pull/3879/changes/86cd8f9a3c937660eb4c0ca345202b0ce21db561 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
