[
https://issues.apache.org/jira/browse/HADOOP-18096?focusedWorklogId=724535&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-724535
]
ASF GitHub Bot logged work on HADOOP-18096:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 10/Feb/22 14:22
Start Date: 10/Feb/22 14:22
Worklog Time Spent: 10m
Work Description: ayushtkn commented on a change in pull request #3940:
URL: https://github.com/apache/hadoop/pull/3940#discussion_r803724330
##########
File path:
hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestDistCpSync.java
##########
@@ -1154,6 +1155,80 @@ public void testSyncSnapshotDiffWithWebHdfs3() throws
Exception {
snapshotDiffWithPaths(webHdfsSource, webHdfsTarget);
}
+ @Test
+ public void testRenameWithFilter() throws Exception {
+ java.nio.file.Path filterFile = null;
+ try {
+ Path sourcePath = new Path(dfs.getWorkingDirectory(), "source");
+
+ // Create some dir inside source
+ dfs.mkdirs(new Path(sourcePath, "dir1"));
+ dfs.mkdirs(new Path(sourcePath, "dir2"));
+
+ // Allow & Create snapshot at source.
+ dfs.allowSnapshot(sourcePath);
+ dfs.createSnapshot(sourcePath, "s1");
+
+ filterFile = Files.createTempFile("filters", "txt");
+ String str = ".*filterDir1.*";
+ try (BufferedWriter writer = new BufferedWriter(
+ new FileWriter(filterFile.toString()))) {
+ writer.write(str);
+ }
+ final DistCpOptions.Builder builder =
+ new DistCpOptions.Builder(new ArrayList<>(Arrays.asList(sourcePath)),
+ target).withFiltersFile(filterFile.toString())
+ .withSyncFolder(true);
+ new DistCp(conf, builder.build()).execute();
+
+ // Check the two directories get copied.
+ ContractTestUtils
+ .assertPathExists(dfs, "dir1 should get copied to target",
+ new Path(target, "dir1"));
+ ContractTestUtils
+ .assertPathExists(dfs, "dir2 should get copied to target",
+ new Path(target, "dir2"));
+
+ // Allow & create initial snapshots on target.
+ dfs.allowSnapshot(target);
+ dfs.createSnapshot(target, "s1");
+
+ // Now do a rename to a filtered name on source.
+ dfs.rename(new Path(sourcePath, "dir1"),
Review comment:
Done, Will push if tests are green
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 724535)
Time Spent: 1h 40m (was: 1.5h)
> Distcp: Sync moves filtered file to home directory rather than deleting
> -----------------------------------------------------------------------
>
> Key: HADOOP-18096
> URL: https://issues.apache.org/jira/browse/HADOOP-18096
> Project: Hadoop Common
> Issue Type: Bug
> Reporter: Ayush Saxena
> Assignee: Ayush Saxena
> Priority: Critical
> Labels: pull-request-available
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> Distcp sync with snapshot, if the file being copied is renamed to a path
> which is in the exclusion filter, tries to delete the file.
> But instead of deleting, it moves the file to home directory
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
