[
https://issues.apache.org/jira/browse/HDFS-13916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16630792#comment-16630792
]
Xiaoyu Yao commented on HDFS-13916:
-----------------------------------
Thanks [~renxunsaky] for reporting and posting the patch. Patch v4 looks pretty
good to me.
I just have a few minor comments:
DistCpSync.java
Line 204: we should check with in case !isRdiff() where the source file system
might not be webhdfs or hdfs.
{code}
else if (fs instanceof WebHdfsFileSystem)
{code}
Line 262: NIT checkstyle (line linger than 80)
TestDistCpSync.java
Line 73-77: NIT: unrelated formatting change can be avoided.
Line 105: same as above, please avoid formatting only change in other places
too.
Line 163-171: initData()/changeData() refactor is not needed as we have a
single cluster and we can always initData with dfs.
Line 311/325: NIT: typo: weather -> whether
Line 839/878: can we refactor the common part of
testSyncSnapshotDiffWithWebHdfs2 and
testSyncSnapshotDiffWithWebHdfs3 into a testHelper to reduce duplicated code?
> Distcp SnapshotDiff not completely implemented for supporting WebHdfs
> ---------------------------------------------------------------------
>
> Key: HDFS-13916
> URL: https://issues.apache.org/jira/browse/HDFS-13916
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: distcp, webhdfs
> Affects Versions: 3.0.1, 3.1.1
> Reporter: Xun REN
> Assignee: Xun REN
> Priority: Major
> Labels: easyfix, newbie, patch
> Attachments: HDFS-13916.002.patch, HDFS-13916.003.patch,
> HDFS-13916.004.patch, HDFS-13916.005.patch, HDFS-13916.patch
>
>
> [~ljain] has worked on the JIRA:
> https://issues.apache.org/jira/browse/HDFS-13052 to provide the possibility
> to make DistCP of SnapshotDiff with WebHDFSFileSystem. However, in the patch,
> there is no modification for the real java class which is used by launching
> the command "hadoop distcp ..."
>
> You can check in the latest version here:
> [https://github.com/apache/hadoop/blob/branch-3.1.1/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpSync.java#L96-L100]
> In the method "preSyncCheck" of the class "DistCpSync", we still check if the
> file system is DFS.
> So I propose to change the class DistCpSync in order to take into
> consideration what was committed by Lokesh Jain.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
