[jira] [Commented] (HDFS-10397) Distcp should ignore -delete option if -diff option is provided instead of exiting
[
https://issues.apache.org/jira/browse/HDFS-10397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15287831#comment-15287831
]
Hudson commented on HDFS-10397:
---
SUCCESS: Integrated in Hadoop-trunk-Commit #9809 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/9809/])
HDFS-10397. Distcp should ignore -delete option if -diff option is (jing9: rev
03788d3015c962eac1a35fa5df39356e8b84731c)
*
hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpOptions.java
*
hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/OptionsParser.java
*
hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestOptionsParser.java
> Distcp should ignore -delete option if -diff option is provided instead of
> exiting
> --
>
> Key: HDFS-10397
> URL: https://issues.apache.org/jira/browse/HDFS-10397
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: distcp
>Affects Versions: 2.8.0
>Reporter: Mingliang Liu
>Assignee: Mingliang Liu
> Fix For: 2.8.0
>
> Attachments: HDFS-10397.000.patch, HDFS-10397.001.patch,
> HDFS-10397.002.patch, HDFS-10397.003.patch, HDFS-10397.003.patch
>
>
> In distcp, {{-delete}} and {{-diff}} options are mutually exclusive.
> [HDFS-8828] brought strictly checking which makes the existing applications
> (or scripts) that work just fine with both {{-delete}} and {{-diff}} options
> previously stop performing because of the
> {{java.lang.IllegalArgumentException: Diff is valid only with update
> options}} exception.
> To make it backward incompatible, we can ignore the {{-delete}} option, given
> {{-diff}} option, instead of exiting the program. Along with that, we can
> print a warning message saying that _Diff is valid only with update options,
> and -delete option is ignored_.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-10397) Distcp should ignore -delete option if -diff option is provided instead of exiting
[
https://issues.apache.org/jira/browse/HDFS-10397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15287823#comment-15287823
]
Mingliang Liu commented on HDFS-10397:
--
Thanks [~jingzhao] and [~yzhangal] for the review and discussion!
> Distcp should ignore -delete option if -diff option is provided instead of
> exiting
> --
>
> Key: HDFS-10397
> URL: https://issues.apache.org/jira/browse/HDFS-10397
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: distcp
>Affects Versions: 2.8.0
>Reporter: Mingliang Liu
>Assignee: Mingliang Liu
> Fix For: 2.8.0
>
> Attachments: HDFS-10397.000.patch, HDFS-10397.001.patch,
> HDFS-10397.002.patch, HDFS-10397.003.patch, HDFS-10397.003.patch
>
>
> In distcp, {{-delete}} and {{-diff}} options are mutually exclusive.
> [HDFS-8828] brought strictly checking which makes the existing applications
> (or scripts) that work just fine with both {{-delete}} and {{-diff}} options
> previously stop performing because of the
> {{java.lang.IllegalArgumentException: Diff is valid only with update
> options}} exception.
> To make it backward incompatible, we can ignore the {{-delete}} option, given
> {{-diff}} option, instead of exiting the program. Along with that, we can
> print a warning message saying that _Diff is valid only with update options,
> and -delete option is ignored_.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-10397) Distcp should ignore -delete option if -diff option is provided instead of exiting
[
https://issues.apache.org/jira/browse/HDFS-10397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15287744#comment-15287744
]
Yongjun Zhang commented on HDFS-10397:
--
Sorry for the delayed response [~liuml07], +1 on rev 003.
> Distcp should ignore -delete option if -diff option is provided instead of
> exiting
> --
>
> Key: HDFS-10397
> URL: https://issues.apache.org/jira/browse/HDFS-10397
> Project: Hadoop HDFS
> Issue Type: Bug
>Affects Versions: 2.8.0
>Reporter: Mingliang Liu
>Assignee: Mingliang Liu
> Attachments: HDFS-10397.000.patch, HDFS-10397.001.patch,
> HDFS-10397.002.patch, HDFS-10397.003.patch, HDFS-10397.003.patch
>
>
> In distcp, {{-delete}} and {{-diff}} options are mutually exclusive.
> [HDFS-8828] brought strictly checking which makes the existing applications
> (or scripts) that work just fine with both {{-delete}} and {{-diff}} options
> previously stop performing because of the
> {{java.lang.IllegalArgumentException: Diff is valid only with update
> options}} exception.
> To make it backward incompatible, we can ignore the {{-delete}} option, given
> {{-diff}} option, instead of exiting the program. Along with that, we can
> print a warning message saying that _Diff is valid only with update options,
> and -delete option is ignored_.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-10397) Distcp should ignore -delete option if -diff option is provided instead of exiting
[
https://issues.apache.org/jira/browse/HDFS-10397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15287202#comment-15287202
]
Hadoop QA commented on HDFS-10397:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 10s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
22s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 15s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 19s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
24s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 12s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
15s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 12s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 12s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s {color} | {color:green} hadoop-tools/hadoop-distcp: patch generated 0 new +
76 unchanged - 11 fixed = 76 total (was 87) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 16s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
8s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
31s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 12s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 9m 5s
{color} | {color:green} hadoop-distcp in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 19m 51s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:2c91fd8 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12804477/HDFS-10397.003.patch |
| JIRA Issue | HDFS-10397 |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 2bed8f9571b2 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 34fddd1 |
| Default Java | 1.8.0_91 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/15462/testReport/ |
| modules | C: hadoop-tools/hadoop-distcp U: hadoop-tools/hadoop-distcp |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/15462/console |
| Powered by | Apache Yetus 0.2.0 http://yetus.apache.org |
This message was automatically generated.
> Distcp should ignore -delete option if -diff option is provided instead of
> exiting
> --
>
> Key: HDFS-10397
> URL: https://issues.apache.org/jira/browse/HDFS-10397
> Project: Hadoop HDFS
> Issue Type: Bug
>Affects Versions: 2.8.0
>Reporter: Mingliang Liu
>Assignee: Mingliang Liu
> Attachments: HDFS-10397.000.patch, HDFS-10397.001.patch,
> HDFS-10397.002.patch, HDFS-10397.003.patch, HDFS-10397.003.patch
>
>
> In
[jira] [Commented] (HDFS-10397) Distcp should ignore -delete option if -diff option is provided instead of exiting
[
https://issues.apache.org/jira/browse/HDFS-10397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285629#comment-15285629
]
Mingliang Liu commented on HDFS-10397:
--
Thanks [~jingzhao] for the review. The latest build and test failures are
caused by some test env misconfiguration when switching to Java 8 (see
[HADOOP-11858]). Let's trigger Jenkins after that is fixed.
> Distcp should ignore -delete option if -diff option is provided instead of
> exiting
> --
>
> Key: HDFS-10397
> URL: https://issues.apache.org/jira/browse/HDFS-10397
> Project: Hadoop HDFS
> Issue Type: Bug
>Affects Versions: 2.8.0
>Reporter: Mingliang Liu
>Assignee: Mingliang Liu
> Attachments: HDFS-10397.000.patch, HDFS-10397.001.patch,
> HDFS-10397.002.patch, HDFS-10397.003.patch
>
>
> In distcp, {{-delete}} and {{-diff}} options are mutually exclusive.
> [HDFS-8828] brought strictly checking which makes the existing applications
> (or scripts) that work just fine with both {{-delete}} and {{-diff}} options
> previously stop performing because of the
> {{java.lang.IllegalArgumentException: Diff is valid only with update
> options}} exception.
> To make it backward incompatible, we can ignore the {{-delete}} option, given
> {{-diff}} option, instead of exiting the program. Along with that, we can
> print a warning message saying that _Diff is valid only with update options,
> and -delete option is ignored_.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-10397) Distcp should ignore -delete option if -diff option is provided instead of exiting
[
https://issues.apache.org/jira/browse/HDFS-10397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285612#comment-15285612
]
Hadoop QA commented on HDFS-10397:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 14s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m 8s
{color} | {color:red} root in trunk failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 13s
{color} | {color:green} trunk passed with JDK v1.8.0_91 {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m 9s
{color} | {color:red} hadoop-distcp in trunk failed with JDK v1.7.0_95. {color}
|
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s {color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m 9s
{color} | {color:red} hadoop-distcp in trunk failed. {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s {color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 9s
{color} | {color:red} hadoop-distcp in trunk failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 10s
{color} | {color:green} trunk passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 12s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m 9s
{color} | {color:red} hadoop-distcp in the patch failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 13s
{color} | {color:green} the patch passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 13s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m 8s
{color} | {color:red} hadoop-distcp in the patch failed with JDK v1.7.0_95.
{color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 8s {color}
| {color:red} hadoop-distcp in the patch failed with JDK v1.7.0_95. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s {color} | {color:green} hadoop-tools/hadoop-distcp: patch generated 0 new +
76 unchanged - 11 fixed = 76 total (was 87) {color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m 10s
{color} | {color:red} hadoop-distcp in the patch failed. {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
11s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 8s
{color} | {color:red} hadoop-distcp in the patch failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 10s
{color} | {color:green} the patch passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 12s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 9m 4s
{color} | {color:green} hadoop-distcp in the patch passed with JDK v1.8.0_91.
{color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 9s {color} |
{color:red} hadoop-distcp in the patch failed with JDK v1.7.0_95. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 13m 39s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:cf2ee45 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12804277/HDFS-10397.003.patch |
| JIRA Issue | HDFS-10397 |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux a30829ef71b4 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
[jira] [Commented] (HDFS-10397) Distcp should ignore -delete option if -diff option is provided instead of exiting
[
https://issues.apache.org/jira/browse/HDFS-10397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285581#comment-15285581
]
Jing Zhao commented on HDFS-10397:
--
+1 on the 003 patch. I will hold the commit to see if [~yzhangal] has further
comments.
> Distcp should ignore -delete option if -diff option is provided instead of
> exiting
> --
>
> Key: HDFS-10397
> URL: https://issues.apache.org/jira/browse/HDFS-10397
> Project: Hadoop HDFS
> Issue Type: Bug
>Affects Versions: 2.8.0
>Reporter: Mingliang Liu
>Assignee: Mingliang Liu
> Attachments: HDFS-10397.000.patch, HDFS-10397.001.patch,
> HDFS-10397.002.patch, HDFS-10397.003.patch
>
>
> In distcp, {{-delete}} and {{-diff}} options are mutually exclusive.
> [HDFS-8828] brought strictly checking which makes the existing applications
> (or scripts) that work just fine with both {{-delete}} and {{-diff}} options
> previously stop performing because of the
> {{java.lang.IllegalArgumentException: Diff is valid only with update
> options}} exception.
> To make it backward incompatible, we can ignore the {{-delete}} option, given
> {{-diff}} option, instead of exiting the program. Along with that, we can
> print a warning message saying that _Diff is valid only with update options,
> and -delete option is ignored_.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-10397) Distcp should ignore -delete option if -diff option is provided instead of exiting
[
https://issues.apache.org/jira/browse/HDFS-10397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285308#comment-15285308
]
Jing Zhao commented on HDFS-10397:
--
Thanks for updating the patch, Mingliang. The current patch looks good to me in
general. One minor:
{code}
580 if (useDiff && deleteMissing) {
581 // -delete and -diff are mutually exclusive. For backward
compatibility,
582 // we ignore the -delete option here, instead of throwing an
583 // IllegalArgumentException. See HDFS-10397 for more discussion.
584 OptionsParser.LOG.warn("-delete and -diff are mutually exclusive.
" +
585 "The -delete option will be ignored.");
586 setDeleteMissing(false);
587 }
{code}
This logic should be moved to the beginning of the {{valid}} method. I.e., the
"-deleteMissing" should be bypassed before it causes any exception along with
some other option settings. A new unit test will also be helpful for this
scenario.
+1 after addressing the comment.
> Distcp should ignore -delete option if -diff option is provided instead of
> exiting
> --
>
> Key: HDFS-10397
> URL: https://issues.apache.org/jira/browse/HDFS-10397
> Project: Hadoop HDFS
> Issue Type: Bug
>Affects Versions: 2.8.0
>Reporter: Mingliang Liu
>Assignee: Mingliang Liu
> Attachments: HDFS-10397.000.patch, HDFS-10397.001.patch,
> HDFS-10397.002.patch
>
>
> In distcp, {{-delete}} and {{-diff}} options are mutually exclusive.
> [HDFS-8828] brought strictly checking which makes the existing applications
> (or scripts) that work just fine with both {{-delete}} and {{-diff}} options
> previously stop performing because of the
> {{java.lang.IllegalArgumentException: Diff is valid only with update
> options}} exception.
> To make it backward incompatible, we can ignore the {{-delete}} option, given
> {{-diff}} option, instead of exiting the program. Along with that, we can
> print a warning message saying that _Diff is valid only with update options,
> and -delete option is ignored_.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-10397) Distcp should ignore -delete option if -diff option is provided instead of exiting
[
https://issues.apache.org/jira/browse/HDFS-10397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15283787#comment-15283787
]
Yongjun Zhang commented on HDFS-10397:
--
Hi [~liuml07],
I think it's a very good idea to do what you did in 002. I will try to review
early next week.
Thanks a lot.
> Distcp should ignore -delete option if -diff option is provided instead of
> exiting
> --
>
> Key: HDFS-10397
> URL: https://issues.apache.org/jira/browse/HDFS-10397
> Project: Hadoop HDFS
> Issue Type: Bug
>Affects Versions: 2.8.0
>Reporter: Mingliang Liu
>Assignee: Mingliang Liu
> Attachments: HDFS-10397.000.patch, HDFS-10397.001.patch,
> HDFS-10397.002.patch
>
>
> In distcp, {{-delete}} and {{-diff}} options are mutually exclusive.
> [HDFS-8828] brought strictly checking which makes the existing applications
> (or scripts) that work just fine with both {{-delete}} and {{-diff}} options
> previously stop performing because of the
> {{java.lang.IllegalArgumentException: Diff is valid only with update
> options}} exception.
> To make it backward incompatible, we can ignore the {{-delete}} option, given
> {{-diff}} option, instead of exiting the program. Along with that, we can
> print a warning message saying that _Diff is valid only with update options,
> and -delete option is ignored_.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-10397) Distcp should ignore -delete option if -diff option is provided instead of exiting
[
https://issues.apache.org/jira/browse/HDFS-10397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15283487#comment-15283487
]
Hadoop QA commented on HDFS-10397:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 24s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
28s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 19s
{color} | {color:green} trunk passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 18s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 23s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
31s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 14s
{color} | {color:green} trunk passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 15s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
19s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 17s
{color} | {color:green} the patch passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 17s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 16s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 16s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s {color} | {color:green} hadoop-tools/hadoop-distcp: patch generated 0 new +
76 unchanged - 11 fixed = 76 total (was 87) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 21s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
11s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
40s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 13s
{color} | {color:green} the patch passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 12s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 9m 2s
{color} | {color:green} hadoop-distcp in the patch passed with JDK v1.8.0_91.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m 44s
{color} | {color:green} hadoop-distcp in the patch passed with JDK v1.7.0_95.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
19s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 32m 8s {color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:cf2ee45 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12804014/HDFS-10397.002.patch |
| JIRA Issue | HDFS-10397 |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 1598bc190d8d 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 3fa1380 |
| Default
[jira] [Commented] (HDFS-10397) Distcp should ignore -delete option if -diff option is provided instead of exiting
[
https://issues.apache.org/jira/browse/HDFS-10397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15283184#comment-15283184
]
Yongjun Zhang commented on HDFS-10397:
--
Hi [~liuml07],
Thanks for the patch. I did a quick browse, and have the following suggestion.
Instead of changing {{validate}} method, I think we can possibly change the
{{OptionParser#parse}} method, something like
{code}
boolean deleteMissing =
command.hasOption(DistCpOptionSwitch.DELETE_MISSING.getSwitch());
boolean diff = command.hasOption(DistCpOptionSwitch.DIFF.getSwitch());
boolean ignoreDeleteMissing = deleteMissing && diff;
if (ignoreDeleteMissing) {
// issue warning message
} else if (deleteMissing) {
//set deleteMissing
}
if (diff) {
// set diff
}
{code}
Basically let the parser to decide whether to ignore some switches, and let
{{DistCpOption#validate}} to decide other invalid situations and throw
InvalidArguement exception. This seems cleaner to me.
What do you think?
Thanks.
> Distcp should ignore -delete option if -diff option is provided instead of
> exiting
> --
>
> Key: HDFS-10397
> URL: https://issues.apache.org/jira/browse/HDFS-10397
> Project: Hadoop HDFS
> Issue Type: Bug
>Affects Versions: 2.8.0
>Reporter: Mingliang Liu
>Assignee: Mingliang Liu
> Attachments: HDFS-10397.000.patch, HDFS-10397.001.patch
>
>
> In distcp, {{-delete}} and {{-diff}} options are mutually exclusive.
> [HDFS-8828] brought strictly checking which makes the existing applications
> (or scripts) that work just fine with both {{-delete}} and {{-diff}} options
> previously stop performing because of the
> {{java.lang.IllegalArgumentException: Diff is valid only with update
> options}} exception.
> To make it backward incompatible, we can ignore the {{-delete}} option, given
> {{-diff}} option, instead of exiting the program. Along with that, we can
> print a warning message saying that _Diff is valid only with update options,
> and -delete option is ignored_.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-10397) Distcp should ignore -delete option if -diff option is provided instead of exiting
[
https://issues.apache.org/jira/browse/HDFS-10397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15282160#comment-15282160
]
Mingliang Liu commented on HDFS-10397:
--
Sample error message:
{code}
2016-05-05
06:41:57,429|beaver.machine|INFO|23004|140021173225216|MainThread|RUNNING:
/usr/hdp/current/hadoop-client/bin/hadoop distcp -update -delete -diff s1 s2
/tmp/source /tmp/target
2016-05-05
06:41:58,461|beaver.machine|INFO|23004|140021173225216|MainThread|16/05/05
06:41:58 ERROR tools.DistCp: Invalid arguments:
2016-05-05
06:41:58,462|beaver.machine|INFO|23004|140021173225216|MainThread|java.lang.IllegalArgumentException:
Diff is valid only with update options
2016-05-05 06:41:58,462|beaver.machine|INFO|23004|140021173225216|MainThread|at
org.apache.hadoop.tools.DistCpOptions.validate(DistCpOptions.java:595)
2016-05-05 06:41:58,462|beaver.machine|INFO|23004|140021173225216|MainThread|at
org.apache.hadoop.tools.DistCpOptions.setUseDiff(DistCpOptions.java:287)
2016-05-05 06:41:58,462|beaver.machine|INFO|23004|140021173225216|MainThread|at
org.apache.hadoop.tools.OptionsParser.parse(OptionsParser.java:235)
2016-05-05 06:41:58,462|beaver.machine|INFO|23004|140021173225216|MainThread|at
org.apache.hadoop.tools.DistCp.run(DistCp.java:115)
2016-05-05 06:41:58,462|beaver.machine|INFO|23004|140021173225216|MainThread|at
org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
2016-05-05 06:41:58,463|beaver.machine|INFO|23004|140021173225216|MainThread|at
org.apache.hadoop.tools.DistCp.main(DistCp.java:453)
2016-05-05
06:41:58,463|beaver.machine|INFO|23004|140021173225216|MainThread|Invalid
arguments: Diff is valid only with update options
{code}
> Distcp should ignore -delete option if -diff option is provided instead of
> exiting
> --
>
> Key: HDFS-10397
> URL: https://issues.apache.org/jira/browse/HDFS-10397
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Mingliang Liu
>Assignee: Mingliang Liu
>
> In distcp, {{-delete}} and {{-diff}} options are mutually exclusive.
> [HDFS-8828] brought strictly checking which makes the existing applications
> (or scripts) that work just fine with both {{-delete}} and {{-diff}} options
> previously stop performing because of the
> {{java.lang.IllegalArgumentException: Diff is valid only with update
> options}} exception.
> To make it backward incompatible, we can ignore the {{-delete}} option, given
> {{-diff}} option, instead of exiting the program. Along with that, we can
> print a warning message saying that _Diff is valid only with update options,
> and -delete option is ignored_.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
