Repository: hadoop
Updated Branches:
  refs/heads/branch-2.7 b8e73cff8 -> 6902789a9


HDFS-9860. Backport HDFS-9638 to branch-2.7. Contributed by Wei-Chiu Chuang.


Project: http://git-wip-us.apache.org/repos/asf/hadoop/repo
Commit: http://git-wip-us.apache.org/repos/asf/hadoop/commit/6902789a
Tree: http://git-wip-us.apache.org/repos/asf/hadoop/tree/6902789a
Diff: http://git-wip-us.apache.org/repos/asf/hadoop/diff/6902789a

Branch: refs/heads/branch-2.7
Commit: 6902789a926eaf48a044c520272c02de4d5202df
Parents: b8e73cf
Author: Akira Ajisaka <aajis...@apache.org>
Authored: Thu Mar 17 02:59:51 2016 +0900
Committer: Akira Ajisaka <aajis...@apache.org>
Committed: Thu Mar 17 02:59:51 2016 +0900

----------------------------------------------------------------------
 hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt                   | 2 ++
 .../main/java/org/apache/hadoop/tools/DistCpOptionSwitch.java | 2 +-
 hadoop-tools/hadoop-distcp/src/site/markdown/DistCp.md.vm     | 7 ++++---
 .../test/java/org/apache/hadoop/tools/TestOptionsParser.java  | 3 ++-
 4 files changed, 9 insertions(+), 5 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/hadoop/blob/6902789a/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
----------------------------------------------------------------------
diff --git a/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt 
b/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
index d29e325..0ed2127 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
+++ b/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
@@ -44,6 +44,8 @@ Release 2.7.3 - UNRELEASED
     HDFS-9048. DistCp documentation is out-of-dated
     (Daisuke Kobayashi via iwasakims)
 
+    HDFS-9860. Backport HDFS-9638 to branch-2.7. (Wei-Chiu Chuang via aajisaka)
+
   OPTIMIZATIONS
 
     HDFS-8845. DiskChecker should not traverse the entire tree (Chang Li via

http://git-wip-us.apache.org/repos/asf/hadoop/blob/6902789a/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpOptionSwitch.java
----------------------------------------------------------------------
diff --git 
a/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpOptionSwitch.java
 
b/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpOptionSwitch.java
index e9c7d46..f11b273 100644
--- 
a/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpOptionSwitch.java
+++ 
b/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpOptionSwitch.java
@@ -82,7 +82,7 @@ public enum DistCpOptionSwitch {
    */
   SSL_CONF(DistCpConstants.CONF_LABEL_SSL_CONF,
       new Option("mapredSslConf", true, "Configuration for ssl config file" +
-          ", to use with hftps://")),
+          ", to use with hftps://. Must be in the classpath.")),
 
   /**
    * Max number of maps to use during copy. DistCp will split work

http://git-wip-us.apache.org/repos/asf/hadoop/blob/6902789a/hadoop-tools/hadoop-distcp/src/site/markdown/DistCp.md.vm
----------------------------------------------------------------------
diff --git a/hadoop-tools/hadoop-distcp/src/site/markdown/DistCp.md.vm 
b/hadoop-tools/hadoop-distcp/src/site/markdown/DistCp.md.vm
index 2f6acb7..93a60bf 100644
--- a/hadoop-tools/hadoop-distcp/src/site/markdown/DistCp.md.vm
+++ b/hadoop-tools/hadoop-distcp/src/site/markdown/DistCp.md.vm
@@ -216,7 +216,7 @@ Command Line Options
 
 Flag              | Description                          | Notes
 ----------------- | ------------------------------------ | --------
-`-p[rbugpcax]` | Preserve r: replication number b: block size u: user g: group 
p: permission c: checksum-type a: ACL x: XAttr | Modification times are not 
preserved. Also, when `-update` is specified, status updates will **not** be 
synchronized unless the file sizes also differ (i.e. unless the file is 
re-created). If -pa is specified, DistCp preserves the permissions also because 
ACLs are a super-set of permissions.
+`-p[rbugpcaxt]` | Preserve r: replication number b: block size u: user g: 
group p: permission c: checksum-type a: ACL x: XAttr t: timestamp | When 
`-update` is specified, status updates will **not** be synchronized unless the 
file sizes also differ (i.e. unless the file is re-created). If -pa is 
specified, DistCp preserves the permissions also because ACLs are a super-set 
of permissions.
 `-i` | Ignore failures | As explained in the Appendix, this option will keep 
more accurate statistics about the copy than the default case. It also 
preserves logs from failed copies, which can be valuable for debugging. 
Finally, a failing map will not cause the job to fail before all splits are 
attempted.
 `-log <logdir>` | Write logs to \<logdir\> | DistCp keeps logs of each file it 
attempts to copy as map output. If a map fails, the log output will not be 
retained if it is re-executed.
 `-m <num_maps>` | Maximum number of simultaneous copies | Specify the number 
of maps to copy data. Note that more maps may not necessarily improve 
throughput.
@@ -231,6 +231,8 @@ Flag              | Description                          | 
Notes
 `-atomic {-tmp <tmp_dir>}` | Specify atomic commit, with optional tmp 
directory. | `-atomic` instructs DistCp to copy the source data to a temporary 
target location, and then move the temporary target to the final-location 
atomically. Data will either be available at final target in a complete and 
consistent form, or not at all. Optionally, `-tmp` may be used to specify the 
location of the tmp-target. If not specified, a default is chosen. **Note:** 
tmp_dir must be on the final target cluster.
 `-mapredSslConf <ssl_conf_file>` | Specify SSL Config file, to be used with 
HSFTP source | When using the hsftp protocol with a source, the security- 
related properties may be specified in a config-file and passed to DistCp. 
\<ssl_conf_file\> needs to be in the classpath.
 `-async` | Run DistCp asynchronously. Quits as soon as the Hadoop Job is 
launched. | The Hadoop Job-id is logged, for tracking.
+`-diff` | Use snapshot diff report to identify the difference between source 
and target. |
+`-skipcrccheck` | Whether to skip CRC checks between source and target paths. |
 
 Architecture of DistCp
 ----------------------
@@ -439,8 +441,7 @@ $H3 SSL Configurations for HSFTP sources
   * `ssl.client.truststore.password`: (Optional) Password for the trust-store
     file.
 
-  The following is an example of the contents of the contents of a SSL
-  Configuration file:
+  The following is an example SSL configuration file:
 
     <configuration>
       <property>

http://git-wip-us.apache.org/repos/asf/hadoop/blob/6902789a/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestOptionsParser.java
----------------------------------------------------------------------
diff --git 
a/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestOptionsParser.java
 
b/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestOptionsParser.java
index cc9da33..45139b4 100644
--- 
a/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestOptionsParser.java
+++ 
b/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestOptionsParser.java
@@ -459,7 +459,7 @@ public class TestOptionsParser {
     Assert.assertFalse(options.shouldPreserve(FileAttribute.XATTR));
 
     options = OptionsParser.parse(new String[] {
-        "-pbrgupcax",
+        "-pbrgupcaxt",
         "-f",
         "hdfs://localhost:8020/source/first",
         "hdfs://localhost:8020/target/"});
@@ -471,6 +471,7 @@ public class TestOptionsParser {
     Assert.assertTrue(options.shouldPreserve(FileAttribute.CHECKSUMTYPE));
     Assert.assertTrue(options.shouldPreserve(FileAttribute.ACL));
     Assert.assertTrue(options.shouldPreserve(FileAttribute.XATTR));
+    Assert.assertTrue(options.shouldPreserve(FileAttribute.TIMES));
 
     options = OptionsParser.parse(new String[] {
         "-pc",

Reply via email to