onebox-li commented on code in PR #3364:
URL: https://github.com/apache/celeborn/pull/3364#discussion_r2212925067
##########
client/src/main/java/org/apache/celeborn/client/read/CelebornInputStream.java:
##########
@@ -526,6 +523,9 @@ private ByteBuf getNextChunk() throws IOException {
throw new CelebornIOException(
"Fetch data from excluded worker! " +
currentReader.getLocation());
}
+ if (!currentReader.hasNext()) {
Review Comment:
I think it is not a good idea to just do this fix.
Which version are you using? If you are using 0.5, this situation may be
caused by inconsistent data between the primary and replica partition. It is
likely that hard splits has occurred when pushing data.
The fix submitted here can really solve the hang task problem, but it may
cause missing data, which is more dangerous.
You can try to turn off either the push replicate or
`celeborn.client.shuffle.batchHandleCommitPartition.enabled` first, which may
solve your problem.
Further solutions may require code modifications. Internally, we are trying
to do some optimizations such as checking commit result and so on.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
