[
https://issues.apache.org/jira/browse/HDFS-15640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yiqun Lin updated HDFS-15640:
-----------------------------
Description: Currently in the DistCpProcedure it must submit distcp round
by round until there is no diff to go to the final distcp stage. The condition
is very strict. During incremental copy stage, if the diff size is under the
given threshold scope then we don't need to wait for no diff. We can start the
final distcp directly. (was: Currently in the DistCpProcedure it must submit
distcp round by round until there is no diff to go to the final distcp stage.
The condition is very strict. If the distcp could finish in an acceptable
period then we don't need to wait for no diff. For example if 3 consecutive
distcp jobs all finish within 10 minutes then we can predict the final distcp
could also finish within 10 minutes. So we can start the final distcp directly.)
> Add snapshot diff threshold to FedBalance
> -----------------------------------------
>
> Key: HDFS-15640
> URL: https://issues.apache.org/jira/browse/HDFS-15640
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Jinglun
> Assignee: Jinglun
> Priority: Major
> Attachments: HDFS-15640.001.patch, HDFS-15640.002.patch,
> HDFS-15640.003.patch, HDFS-15640.004.patch
>
>
> Currently in the DistCpProcedure it must submit distcp round by round until
> there is no diff to go to the final distcp stage. The condition is very
> strict. During incremental copy stage, if the diff size is under the given
> threshold scope then we don't need to wait for no diff. We can start the
> final distcp directly.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
