[ https://issues.apache.org/jira/browse/HDFS-15640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17220448#comment-17220448 ]
Yiqun Lin commented on HDFS-15640: ---------------------------------- Thanks for updating the patch, [~LiJinglun]! Looks very great now. Catch one comment is outdated: {code:java} + * @return true if moving to the next stage. false if the conditions are not + * satisfied. + * @throws RetryException if the conditions are not satisfied and there is no + * diff needed to be copied.x + */ + @VisibleForTesting + boolean diffDistCpStageDone() throws IOException, RetryException { {code} Please update {noformat} ...and there is no diff needed to be copied.. {noformat} to {noformat} ...and the diff size is under the given threshold scope.. {noformat} +1 once this addressed. > RBF: Add fast distcp threshold to FedBalance. > --------------------------------------------- > > Key: HDFS-15640 > URL: https://issues.apache.org/jira/browse/HDFS-15640 > Project: Hadoop HDFS > Issue Type: Sub-task > Reporter: Jinglun > Assignee: Jinglun > Priority: Major > Attachments: HDFS-15640.001.patch, HDFS-15640.002.patch, > HDFS-15640.003.patch > > > Currently in the DistCpProcedure it must submit distcp round by round until > there is no diff to go to the final distcp stage. The condition is very > strict. If the distcp could finish in an acceptable period then we don't need > to wait for no diff. For example if 3 consecutive distcp jobs all finish > within 10 minutes then we can predict the final distcp could also finish > within 10 minutes. So we can start the final distcp directly. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org