[jira] [Commented] (HADOOP-9022) Hadoop distcp tool fails to copy file if -m 0 specified
[ https://issues.apache.org/jira/browse/HADOOP-9022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13496099#comment-13496099 ] Hudson commented on HADOOP-9022: Integrated in Hadoop-Yarn-trunk #35 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/35/]) HADOOP-9022. Hadoop distcp tool fails to copy file if -m 0 specified (Jonathan Eagles vai bobby) (Revision 1408488) Result = SUCCESS bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1408488 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpOptions.java * /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestOptionsParser.java Hadoop distcp tool fails to copy file if -m 0 specified --- Key: HADOOP-9022 URL: https://issues.apache.org/jira/browse/HADOOP-9022 Project: Hadoop Common Issue Type: Bug Affects Versions: 0.23.1, 0.23.3, 0.23.4 Reporter: Haiyang Jiang Assignee: Jonathan Eagles Fix For: 3.0.0, 2.0.3-alpha, 0.23.5 Attachments: HADOOP-9022.patch When trying to copy file using distcp on H23, if -m 0 is specified, distcp will just spawn 0 mapper tasks and the file will not be copied. But this used to work before H23, even when -m 0 specified, distcp will always copy the files. Checked the code of DistCp.java Before the rewrite, it set the number maps at least to 1 job.setNumMapTasks(Math.max(numMaps, 1)); But in the newest code, it just takes the input from user: job.getConfiguration().set(JobContext.NUM_MAPS, String.valueOf(inputOptions.getMaxMaps())); -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9022) Hadoop distcp tool fails to copy file if -m 0 specified
[ https://issues.apache.org/jira/browse/HADOOP-9022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13496149#comment-13496149 ] Hudson commented on HADOOP-9022: Integrated in Hadoop-Hdfs-0.23-Build #434 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/434/]) svn merge -c 1408488 FIXES: HADOOP-9022. Hadoop distcp tool fails to copy file if -m 0 specified (Jonathan Eagles vai bobby) (Revision 1408493) Result = SUCCESS bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1408493 Files : * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpOptions.java * /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestOptionsParser.java Hadoop distcp tool fails to copy file if -m 0 specified --- Key: HADOOP-9022 URL: https://issues.apache.org/jira/browse/HADOOP-9022 Project: Hadoop Common Issue Type: Bug Affects Versions: 0.23.1, 0.23.3, 0.23.4 Reporter: Haiyang Jiang Assignee: Jonathan Eagles Fix For: 3.0.0, 2.0.3-alpha, 0.23.5 Attachments: HADOOP-9022.patch When trying to copy file using distcp on H23, if -m 0 is specified, distcp will just spawn 0 mapper tasks and the file will not be copied. But this used to work before H23, even when -m 0 specified, distcp will always copy the files. Checked the code of DistCp.java Before the rewrite, it set the number maps at least to 1 job.setNumMapTasks(Math.max(numMaps, 1)); But in the newest code, it just takes the input from user: job.getConfiguration().set(JobContext.NUM_MAPS, String.valueOf(inputOptions.getMaxMaps())); -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9022) Hadoop distcp tool fails to copy file if -m 0 specified
[ https://issues.apache.org/jira/browse/HADOOP-9022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13496174#comment-13496174 ] Hudson commented on HADOOP-9022: Integrated in Hadoop-Hdfs-trunk #1225 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1225/]) HADOOP-9022. Hadoop distcp tool fails to copy file if -m 0 specified (Jonathan Eagles vai bobby) (Revision 1408488) Result = SUCCESS bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1408488 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpOptions.java * /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestOptionsParser.java Hadoop distcp tool fails to copy file if -m 0 specified --- Key: HADOOP-9022 URL: https://issues.apache.org/jira/browse/HADOOP-9022 Project: Hadoop Common Issue Type: Bug Affects Versions: 0.23.1, 0.23.3, 0.23.4 Reporter: Haiyang Jiang Assignee: Jonathan Eagles Fix For: 3.0.0, 2.0.3-alpha, 0.23.5 Attachments: HADOOP-9022.patch When trying to copy file using distcp on H23, if -m 0 is specified, distcp will just spawn 0 mapper tasks and the file will not be copied. But this used to work before H23, even when -m 0 specified, distcp will always copy the files. Checked the code of DistCp.java Before the rewrite, it set the number maps at least to 1 job.setNumMapTasks(Math.max(numMaps, 1)); But in the newest code, it just takes the input from user: job.getConfiguration().set(JobContext.NUM_MAPS, String.valueOf(inputOptions.getMaxMaps())); -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9022) Hadoop distcp tool fails to copy file if -m 0 specified
[ https://issues.apache.org/jira/browse/HADOOP-9022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13496207#comment-13496207 ] Hudson commented on HADOOP-9022: Integrated in Hadoop-Mapreduce-trunk #1256 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1256/]) HADOOP-9022. Hadoop distcp tool fails to copy file if -m 0 specified (Jonathan Eagles vai bobby) (Revision 1408488) Result = SUCCESS bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1408488 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpOptions.java * /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestOptionsParser.java Hadoop distcp tool fails to copy file if -m 0 specified --- Key: HADOOP-9022 URL: https://issues.apache.org/jira/browse/HADOOP-9022 Project: Hadoop Common Issue Type: Bug Affects Versions: 0.23.1, 0.23.3, 0.23.4 Reporter: Haiyang Jiang Assignee: Jonathan Eagles Fix For: 3.0.0, 2.0.3-alpha, 0.23.5 Attachments: HADOOP-9022.patch When trying to copy file using distcp on H23, if -m 0 is specified, distcp will just spawn 0 mapper tasks and the file will not be copied. But this used to work before H23, even when -m 0 specified, distcp will always copy the files. Checked the code of DistCp.java Before the rewrite, it set the number maps at least to 1 job.setNumMapTasks(Math.max(numMaps, 1)); But in the newest code, it just takes the input from user: job.getConfiguration().set(JobContext.NUM_MAPS, String.valueOf(inputOptions.getMaxMaps())); -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9022) Hadoop distcp tool fails to copy file if -m 0 specified
[ https://issues.apache.org/jira/browse/HADOOP-9022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13495673#comment-13495673 ] Robert Joseph Evans commented on HADOOP-9022: - The patch looks fine to me. I am +1 pending Jenkins. Hadoop distcp tool fails to copy file if -m 0 specified --- Key: HADOOP-9022 URL: https://issues.apache.org/jira/browse/HADOOP-9022 Project: Hadoop Common Issue Type: Bug Affects Versions: 0.23.1, 0.23.3, 0.23.4 Reporter: Haiyang Jiang Assignee: Jonathan Eagles Attachments: HADOOP-9022.patch When trying to copy file using distcp on H23, if -m 0 is specified, distcp will just spawn 0 mapper tasks and the file will not be copied. But this used to work before H23, even when -m 0 specified, distcp will always copy the files. Checked the code of DistCp.java Before the rewrite, it set the number maps at least to 1 job.setNumMapTasks(Math.max(numMaps, 1)); But in the newest code, it just takes the input from user: job.getConfiguration().set(JobContext.NUM_MAPS, String.valueOf(inputOptions.getMaxMaps())); -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9022) Hadoop distcp tool fails to copy file if -m 0 specified
[
https://issues.apache.org/jira/browse/HADOOP-9022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13495686#comment-13495686
]
Hadoop QA commented on HADOOP-9022:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12553197/HADOOP-9022.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-tools/hadoop-distcp:
org.apache.hadoop.tools.TestCopyListing
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1732//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1732//console
This message is automatically generated.
Hadoop distcp tool fails to copy file if -m 0 specified
---
Key: HADOOP-9022
URL: https://issues.apache.org/jira/browse/HADOOP-9022
Project: Hadoop Common
Issue Type: Bug
Affects Versions: 0.23.1, 0.23.3, 0.23.4
Reporter: Haiyang Jiang
Assignee: Jonathan Eagles
Attachments: HADOOP-9022.patch
When trying to copy file using distcp on H23, if -m 0 is specified, distcp
will just spawn 0 mapper tasks and the file will not be copied.
But this used to work before H23, even when -m 0 specified, distcp will
always copy the files.
Checked the code of DistCp.java
Before the rewrite, it set the number maps at least to 1
job.setNumMapTasks(Math.max(numMaps, 1));
But in the newest code, it just takes the input from user:
job.getConfiguration().set(JobContext.NUM_MAPS,
String.valueOf(inputOptions.getMaxMaps()));
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9022) Hadoop distcp tool fails to copy file if -m 0 specified
[ https://issues.apache.org/jira/browse/HADOOP-9022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13495710#comment-13495710 ] Robert Joseph Evans commented on HADOOP-9022: - The test failure seems to be happening with or without the patch. I filed HADOOP-9025 for it. Hadoop distcp tool fails to copy file if -m 0 specified --- Key: HADOOP-9022 URL: https://issues.apache.org/jira/browse/HADOOP-9022 Project: Hadoop Common Issue Type: Bug Affects Versions: 0.23.1, 0.23.3, 0.23.4 Reporter: Haiyang Jiang Assignee: Jonathan Eagles Attachments: HADOOP-9022.patch When trying to copy file using distcp on H23, if -m 0 is specified, distcp will just spawn 0 mapper tasks and the file will not be copied. But this used to work before H23, even when -m 0 specified, distcp will always copy the files. Checked the code of DistCp.java Before the rewrite, it set the number maps at least to 1 job.setNumMapTasks(Math.max(numMaps, 1)); But in the newest code, it just takes the input from user: job.getConfiguration().set(JobContext.NUM_MAPS, String.valueOf(inputOptions.getMaxMaps())); -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9022) Hadoop distcp tool fails to copy file if -m 0 specified
[ https://issues.apache.org/jira/browse/HADOOP-9022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13495725#comment-13495725 ] Hudson commented on HADOOP-9022: Integrated in Hadoop-trunk-Commit #3005 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/3005/]) HADOOP-9022. Hadoop distcp tool fails to copy file if -m 0 specified (Jonathan Eagles vai bobby) (Revision 1408488) Result = SUCCESS bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1408488 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpOptions.java * /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestOptionsParser.java Hadoop distcp tool fails to copy file if -m 0 specified --- Key: HADOOP-9022 URL: https://issues.apache.org/jira/browse/HADOOP-9022 Project: Hadoop Common Issue Type: Bug Affects Versions: 0.23.1, 0.23.3, 0.23.4 Reporter: Haiyang Jiang Assignee: Jonathan Eagles Fix For: 3.0.0, 2.0.3-alpha, 0.23.5 Attachments: HADOOP-9022.patch When trying to copy file using distcp on H23, if -m 0 is specified, distcp will just spawn 0 mapper tasks and the file will not be copied. But this used to work before H23, even when -m 0 specified, distcp will always copy the files. Checked the code of DistCp.java Before the rewrite, it set the number maps at least to 1 job.setNumMapTasks(Math.max(numMaps, 1)); But in the newest code, it just takes the input from user: job.getConfiguration().set(JobContext.NUM_MAPS, String.valueOf(inputOptions.getMaxMaps())); -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9022) Hadoop distcp tool fails to copy file if -m 0 specified
[ https://issues.apache.org/jira/browse/HADOOP-9022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13494685#comment-13494685 ] Allen Wittenauer commented on HADOOP-9022: -- When -m 0 is specified, it should probably just call copy in a serial fashion across the list and not run a MapReduce job. After all, setting the number of maps to zero implies that one doesn't want a job executed at all. Hadoop distcp tool fails to copy file if -m 0 specified --- Key: HADOOP-9022 URL: https://issues.apache.org/jira/browse/HADOOP-9022 Project: Hadoop Common Issue Type: Bug Affects Versions: 0.23.1, 0.23.3, 0.23.4 Reporter: Haiyang Jiang When trying to copy file using distcp on H23, if -m 0 is specified, distcp will just spawn 0 mapper tasks and the file will not be copied. But this used to work before H23, even when -m 0 specified, distcp will always copy the files. Checked the code of DistCp.java Before the rewrite, it set the number maps at least to 1 job.setNumMapTasks(Math.max(numMaps, 1)); But in the newest code, it just takes the input from user: job.getConfiguration().set(JobContext.NUM_MAPS, String.valueOf(inputOptions.getMaxMaps())); -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
