Github user cloud-fan commented on a diff in the pull request:

    https://github.com/apache/spark/pull/19788#discussion_r169228298
  
    --- Diff: 
common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/ExternalShuffleBlockResolver.java
 ---
    @@ -157,21 +157,34 @@ public void registerExecutor(
       }
     
       /**
    -   * Obtains a FileSegmentManagedBuffer from (shuffleId, mapId, reduceId). 
We make assumptions
    +   * Obtains a FileSegmentManagedBuffer from (shuffleId, mapId, reduceId, 
numBlocks). We make assumptions
        * about how the hash and sort based shuffles store their data.
        */
       public ManagedBuffer getBlockData(
           String appId,
           String execId,
           int shuffleId,
           int mapId,
    -      int reduceId) {
    +      int reduceId,
    +      int numBlocks) {
         ExecutorShuffleInfo executor = executors.get(new AppExecId(appId, 
execId));
         if (executor == null) {
           throw new RuntimeException(
             String.format("Executor is not registered (appId=%s, execId=%s)", 
appId, execId));
         }
    -    return getSortBasedShuffleBlockData(executor, shuffleId, mapId, 
reduceId);
    +    return getSortBasedShuffleBlockData(executor, shuffleId, mapId, 
reduceId, numBlocks);
    +  }
    +
    +  /**
    +   * This interface is for backward compatible.
    +   */
    +  public ManagedBuffer getBlockData(
    --- End diff --
    
    where will we call this method?


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

Reply via email to