Yongjun Zhang commented on HDFS-10314:

BTW [~jingzhao],

The usage of the solution is:

1. distsync -diff s1 s2 src tgt

This will be exactly the same behavior as "distcp -diff s1 s2 src tgt", it will 
copy the delta changes made in src (between snapshot s1 and s2) to target.

It requires that snapshot s1 to be older than s2

2. distsync -rdiff s2 s1 src tgt

snapshot s2 has to be newer than s1; 

That is, since the creation of s1 on tgt, some changes are made on tgt, and 
these changes are captured in s2.

The goal of this command is to make tgt go back to s1, by reverting the changes 
between s1 and s2, and copy the necessary files from s1 of src.

The src could be the same as tgt, or different as we discussed earlier. When 
the src is different than tgt, src's s1 is expected to be the exact content as 
s1 of tgt. So we can copy files from either s1 of src, or s1 of tgt.

Would you please take a look?

Thanks a lot!

> A new tool to sync current HDFS view to specified snapshot
> ----------------------------------------------------------
>                 Key: HDFS-10314
>                 URL: https://issues.apache.org/jira/browse/HDFS-10314
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: tools
>            Reporter: Yongjun Zhang
>            Assignee: Yongjun Zhang
>         Attachments: HDFS-10314.001.patch
> HDFS-9820 proposed adding -rdiff switch to distcp, as a reversed operation of 
> -diff switch. 
> Upon discussion with [~jingzhao], we will introduce a new tool that wraps 
> around distcp to achieve the same purpose.
> I'm thinking about calling the new tool "rsync", similar to unix/linux 
> command "rsync". The "r" here means remote.
> The syntax that simulate -rdiff behavior proposed in HDFS-9820 is
> {code}
> rsync <fromSnapshotName>  <toSnapshotName>  <source> <target>
> {code}
> This command ensure <fromSnapshotName>  is newer than <toSnapshotName>.
> I think, In the future, we can add another command to have the functionality 
> of -diff switch of distcp.
> {code}
> sync <fromSnapshotName>  <toSnapshotName>  <source> <target>
> {code}
> that ensures <fromSnapshotName>  is older than <toSnapshotName>.
> Thanks [~jingzhao].

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

Reply via email to