akpatnam25 commented on PR #2064: URL: https://github.com/apache/incubator-celeborn/pull/2064#issuecomment-1803395072
> Hi @akpatnam25 , thanks for this PR! I left some comments, and I think some of them are hard to address in current design. Here is my thought: > > This PR changes the barrier to send ACK. Previously ACK is sent when data reaches primary and replica's memory, this PR wants to send ACK after all `fileWriter.write(body)`s are called and check whether there is Exception. > > I‘m fine with this intention. For implementation, how about this: > > 1. When `doReplicate == false`, invoke `callback` after the file write logic > 2. When `doReplicate == true`, keep the current code structure, but > 2.1 Set status to `SUCCESS` or `FAIL`(with exception) after the file write logic > 2.2 In `wrappedCallback#onSuccess` and `wrappedCallback#onFailure`, wait for the status to be available before sending ACK (in current PR it will wait for Future complete anyway). > > WDYT? cc @mridulm @otterc @rmcyang @waitinfuture I've updated to use this new implementation that you have suggested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
