ewoodbury commented on code in PR #3879:
URL: https://github.com/apache/celeborn/pull/3879#discussion_r4331530132
##########
worker/src/main/scala/org/apache/celeborn/service/deploy/worker/storage/TierWriter.scala:
##########
@@ -691,39 +699,36 @@ class DfsTierWriter(
}
}
+ // A copy whose data stream or index failed to close is unreadable, so it
must not count as
+ // committed. S3/OSS upload completion failures are unchecked and already
fail the commit.
+ override protected def failCommitOnCloseStreamsError: Boolean = true
Review Comment:
Thanks - addressed in:
https://github.com/apache/celeborn/commit/86cd8f9a3c937660eb4c0ca345202b0ce21db561,
so these stream-close failures get propagated, covered by a regression test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
