Rakesh R commented on HDFS-10884:

Thanks [~umamaheswararao] for the initial feedback.

bq. Are you planning to notify for each and every block or all blocks 
associated to trackID as combined result?
Good point. Following is the logic where it invokes the handler. I need to do 
few changes in this area and invoke handler only once for updating the combined 
status of block movements under one trackId. I think, I could move the 
invocation if {{blocksMoving.isEmpty()}}, which means there is no inprogress 
block movement for this trackId. I will modify this logic in the next patch. 
How about renaming the handler class to {{BlockMovementsCompletionHandler}} 
representing collection of block movements under the respective {{trackId}}?
+  // Keeps the information - trackID vs its list of blocks
+  private final Map<Long, List<Future<BlockMovementResult>>> moverTaskFutures;
+          if (blocksMoving.isEmpty()) {
+            synchronized (moverTaskFutures) {
+              moverTaskFutures.remove(result.getTrackId());
+            }
+          }
+          LOG.debug("Completed block movement. {}", result);
+          completionHandler.handle(result); // handle completed block 

> [SPS]: Add block movement tracker to track the completion of block movement 
> future tasks at DN
> ----------------------------------------------------------------------------------------------
>                 Key: HDFS-10884
>                 URL: https://issues.apache.org/jira/browse/HDFS-10884
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: datanode
>    Affects Versions: HDFS-10285
>            Reporter: Rakesh R
>            Assignee: Rakesh R
>         Attachments: HDFS-10884-HDFS-10285-00.patch, 
> HDFS-10884-HDFS-10285-01.patch, HDFS-10884-HDFS-10285-02.patch, 
> HDFS-10884-HDFS-10285-03.patch, HDFS-10884-HDFS-10285-04.patch
> Presently 
> [StoragePolicySatisfyWorker#processBlockMovingTasks()|https://github.com/apache/hadoop/blob/HDFS-10285/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/StoragePolicySatisfyWorker.java#L147]
>  function act as a blocking call. The idea of this jira is to implement a 
> mechanism to track these movements async so that would allow other movement 
> while processing the previous one.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

Reply via email to