FMX commented on code in PR #3115:
URL: https://github.com/apache/celeborn/pull/3115#discussion_r1969161015
##########
client/src/main/java/org/apache/celeborn/client/read/DfsPartitionReader.java:
##########
@@ -122,7 +126,7 @@ public DfsPartitionReader(
e);
}
- if (endMapIndex != Integer.MAX_VALUE) {
+ if (endMapIndex != Integer.MAX_VALUE && endMapIndex != -1) {
Review Comment:
If the feature of optimizing skew partition read has been enabled, there
will be no more sort. So here should not read the sorted DFS file.
The condition should be the condition startMapIndex > endMapIndex to be true
and read chunk range from DFS shuffle file.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
