Github user jiangxb1987 commented on a diff in the pull request:
https://github.com/apache/spark/pull/19788#discussion_r152976135
--- Diff:
common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/ExternalShuffleBlockResolver.java
---
@@ -165,13 +165,23 @@ public ManagedBuffer getBlockData(
String execId,
int shuffleId,
int mapId,
- int reduceId) {
+ int reduceId,
+ int length) {
ExecutorShuffleInfo executor = executors.get(new AppExecId(appId,
execId));
if (executor == null) {
throw new RuntimeException(
String.format("Executor is not registered (appId=%s, execId=%s)",
appId, execId));
}
- return getSortBasedShuffleBlockData(executor, shuffleId, mapId,
reduceId);
+ return getSortBasedShuffleBlockData(executor, shuffleId, mapId,
reduceId, length);
+ }
+
+ public ManagedBuffer getBlockData(
--- End diff --
nit: we should move the original comment here, and explain the different
usages of these two functions.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
