[jira] Commented: (MAPREDUCE-548) Global scheduling in the Fair Scheduler
[ https://issues.apache.org/jira/browse/MAPREDUCE-548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12727154#action_12727154 ] Hadoop QA commented on MAPREDUCE-548: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12412509/mapreduce-548-v1.patch against trunk revision 790971. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/348/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/348/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/348/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/348/console This message is automatically generated. Global scheduling in the Fair Scheduler --- Key: MAPREDUCE-548 URL: https://issues.apache.org/jira/browse/MAPREDUCE-548 Project: Hadoop Map/Reduce Issue Type: New Feature Reporter: Matei Zaharia Assignee: Matei Zaharia Fix For: 0.21.0 Attachments: fs-global-v0.patch, hadoop-4667-v1.patch, hadoop-4667-v1b.patch, hadoop-4667-v2.patch, HADOOP-4667_api.patch, mapreduce-548-v1.patch, mapreduce-548.patch The current schedulers in Hadoop all examine a single job on every heartbeat when choosing which tasks to assign, choosing the job based on FIFO or fair sharing. There are inherent limitations to this approach. For example, if the job at the front of the queue is small (e.g. 10 maps, in a cluster of 100 nodes), then on average it will launch only one local map on the first 10 heartbeats while it is at the head of the queue. This leads to very poor locality for small jobs. Instead, we need a more global view of scheduling that can look at multiple jobs. To resolve the locality problem, we will use the following algorithm: - If the job at the head of the queue has no node-local task to launch, skip it and look through other jobs. - If a job has waited at least T1 seconds while being skipped, also allow it to launch rack-local tasks. - If a job has waited at least T2 T1 seconds, also allow it to launch off-rack tasks. This algorithm improves locality while bounding the delay that any job experiences in launching a task. It turns out that whether waiting is useful depends on how many tasks are left in the job - the probability of getting a heartbeat from a node with a local task - and on whether the job is CPU or IO bound. Thus there may be logic for removing the wait on the last few tasks in the job. As a related issue, once we allow global scheduling, we can launch multiple tasks per heartbeat, as in HADOOP-3136. The initial implementation of HADOOP-3136 adversely affected performance because it only launched multiple tasks from the same job, but with the wait rule above, we will only do this for jobs that are allowed to launch non-local tasks. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-679) XML-based metrics as JSP servlet for JobTracker
[ https://issues.apache.org/jira/browse/MAPREDUCE-679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12727177#action_12727177 ] Hadoop QA commented on MAPREDUCE-679: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12412525/MAPREDUCE-679.2.patch against trunk revision 790971. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 2 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/351/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/351/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/351/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/351/console This message is automatically generated. XML-based metrics as JSP servlet for JobTracker --- Key: MAPREDUCE-679 URL: https://issues.apache.org/jira/browse/MAPREDUCE-679 Project: Hadoop Map/Reduce Issue Type: New Feature Components: jobtracker Reporter: Aaron Kimball Assignee: Aaron Kimball Attachments: MAPREDUCE-679.2.patch, MAPREDUCE-679.patch In HADOOP-4559, a general REST API for reporting metrics was proposed but work seems to have stalled. In the interim, we have a simple XML translation of the existing JobTracker status page which provides the same metrics (including the tables of running/completed/failed jobs) as the human-readable page. This is a relatively lightweight addition to provide some machine-understandable metrics reporting. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-685) Sqoop will fail with OutOfMemory on large tables using mysql
[ https://issues.apache.org/jira/browse/MAPREDUCE-685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12727224#action_12727224 ] Hadoop QA commented on MAPREDUCE-685: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12412298/MAPREDUCE-685.patch.2 against trunk revision 790971. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 6 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/354/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/354/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/354/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/354/console This message is automatically generated. Sqoop will fail with OutOfMemory on large tables using mysql Key: MAPREDUCE-685 URL: https://issues.apache.org/jira/browse/MAPREDUCE-685 Project: Hadoop Map/Reduce Issue Type: Bug Components: contrib/sqoop Reporter: Aaron Kimball Assignee: Aaron Kimball Attachments: MAPREDUCE-685.patch, MAPREDUCE-685.patch.2 The default MySQL JDBC client behavior is to buffer the entire ResultSet in the client before allowing the user to use the ResultSet object. On large SELECTs, this can cause OutOfMemory exceptions, even when the client intends to close the ResultSet after reading only a few rows. The MySQL ConnManager should configure its connection to use row-at-a-time delivery of results to the client. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-710) Sqoop should read and transmit passwords in a more secure manner
[ https://issues.apache.org/jira/browse/MAPREDUCE-710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12727764#action_12727764 ] Hadoop QA commented on MAPREDUCE-710: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12412634/MAPREDUCE-710.patch against trunk revision 791418. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 8 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/357/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/357/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/357/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/357/console This message is automatically generated. Sqoop should read and transmit passwords in a more secure manner Key: MAPREDUCE-710 URL: https://issues.apache.org/jira/browse/MAPREDUCE-710 Project: Hadoop Map/Reduce Issue Type: Improvement Components: contrib/sqoop Reporter: Aaron Kimball Assignee: Aaron Kimball Attachments: MAPREDUCE-710.patch Sqoop's current support for passwords involves reading passwords from the command line --password foo, which makes the password visible to other users via 'ps'. An invisible-console approach should be taken. Related, Sqoop transmits passwords to mysqldump in the same fashion, which is also insecure. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-546) Provide sample fair scheduler config file in conf/ and set config file property to point to this by default
[ https://issues.apache.org/jira/browse/MAPREDUCE-546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12727989#action_12727989 ] Hadoop QA commented on MAPREDUCE-546: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12412664/mapreduce-546.patch against trunk revision 791418. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/360/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/360/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/360/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/360/console This message is automatically generated. Provide sample fair scheduler config file in conf/ and set config file property to point to this by default --- Key: MAPREDUCE-546 URL: https://issues.apache.org/jira/browse/MAPREDUCE-546 Project: Hadoop Map/Reduce Issue Type: New Feature Reporter: Matei Zaharia Priority: Minor Fix For: 0.21.0 Attachments: mapreduce-546.patch The capacity scheduler includes a config file template in hadoop/conf, so it would make sense to create a similar one for the fair scheduler and mention it in the README. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-716) org.apache.hadoop.mapred.lib.db.DBInputformat not working with oracle
[ https://issues.apache.org/jira/browse/MAPREDUCE-716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12728092#action_12728092 ] Hadoop QA commented on MAPREDUCE-716: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12412687/HADOOP-5482.trunk.patch against trunk revision 791739. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/361/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/361/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/361/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/361/console This message is automatically generated. org.apache.hadoop.mapred.lib.db.DBInputformat not working with oracle - Key: MAPREDUCE-716 URL: https://issues.apache.org/jira/browse/MAPREDUCE-716 Project: Hadoop Map/Reduce Issue Type: Bug Environment: Java 1.6, HAdoop0.19.0, Linux..Oracle, Reporter: evanand Attachments: HADOOP-5482.20-branch.patch, HADOOP-5482.patch, HADOOP-5482.trunk.patch Original Estimate: 24h Remaining Estimate: 24h org.apache.hadoop.mapred.lib.db.DBInputformat not working with oracle. The out of the box implementation of the Hadoop is working properly with mysql/hsqldb, but NOT with oracle. Reason is DBInputformat is implemented with mysql/hsqldb specific query constructs like LIMIT, OFFSET. FIX: building a database provider specific logic based on the database providername (which we can get using connection). I HAVE ALREADY IMPLEMENTED IT FOR ORACLE...READY TO CHECK_IN CODE -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-713) Sqoop has some superfluous imports
[ https://issues.apache.org/jira/browse/MAPREDUCE-713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12728442#action_12728442 ] Hadoop QA commented on MAPREDUCE-713: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12412765/MAPREDUCE-713.2.patch against trunk revision 791909. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 6 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/363/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/363/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/363/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/363/console This message is automatically generated. Sqoop has some superfluous imports -- Key: MAPREDUCE-713 URL: https://issues.apache.org/jira/browse/MAPREDUCE-713 Project: Hadoop Map/Reduce Issue Type: Improvement Components: contrib/sqoop Reporter: Aaron Kimball Assignee: Aaron Kimball Priority: Trivial Attachments: MAPREDUCE-713.2.patch, MAPREDUCE-713.patch Some classes have vestigial imports that should be removed -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-546) Provide sample fair scheduler config file in conf/ and set config file property to point to this by default
[ https://issues.apache.org/jira/browse/MAPREDUCE-546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12728541#action_12728541 ] Hadoop QA commented on MAPREDUCE-546: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12412768/mapreduce-546-v1.patch against trunk revision 791909. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/365/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/365/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/365/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/365/console This message is automatically generated. Provide sample fair scheduler config file in conf/ and set config file property to point to this by default --- Key: MAPREDUCE-546 URL: https://issues.apache.org/jira/browse/MAPREDUCE-546 Project: Hadoop Map/Reduce Issue Type: New Feature Reporter: Matei Zaharia Priority: Minor Fix For: 0.21.0 Attachments: mapreduce-546-v1.patch, mapreduce-546.patch The capacity scheduler includes a config file template in hadoop/conf, so it would make sense to create a similar one for the fair scheduler and mention it in the README. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-717) Fix some corner case issues in speculative execution (post hadoop-2141)
[ https://issues.apache.org/jira/browse/MAPREDUCE-717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12729413#action_12729413 ] Hadoop QA commented on MAPREDUCE-717: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12412839/717.patch against trunk revision 792613. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/368/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/368/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/368/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/368/console This message is automatically generated. Fix some corner case issues in speculative execution (post hadoop-2141) --- Key: MAPREDUCE-717 URL: https://issues.apache.org/jira/browse/MAPREDUCE-717 Project: Hadoop Map/Reduce Issue Type: Bug Components: jobtracker Affects Versions: 0.21.0 Reporter: Devaraj Das Assignee: Devaraj Das Fix For: 0.21.0 Attachments: 717.patch Some corner case issues can be fixed: 1) Setup task should not add anything to the job statistics (since they are really fast and might affect the statistics of a job with few tasks) 2) The statistics computations should be guarded for cases where things like sumOfSquares could become less than zero (due to rounding errors mostly). 3) The method TaskInProgress.getCurrentProgressRate() should take into account the COMMIT_PENDING state 4) The testcase TestSpeculativeExecution.testTaskLATEScheduling could be made more robust -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-741) New Hadoop MapReduce Site
[ https://issues.apache.org/jira/browse/MAPREDUCE-741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12729512#action_12729512 ] Hadoop QA commented on MAPREDUCE-741: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12412939/mapreduce-741.patch against trunk revision 792704. +1 @author. The patch does not contain any @author tags. +0 tests included. The patch appears to be a documentation patch that doesn't require tests. -1 patch. The patch command could not apply the patch. Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/371/console This message is automatically generated. New Hadoop MapReduce Site - Key: MAPREDUCE-741 URL: https://issues.apache.org/jira/browse/MAPREDUCE-741 Project: Hadoop Map/Reduce Issue Type: Task Components: documentation Reporter: Corinne Chandel Attachments: mapreduce-741.patch New Hadoop MapReduce Site Set up site (initial pass). May need to add more content. May need to update some links. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-383) pipes combiner does not reset properly after a spill
[ https://issues.apache.org/jira/browse/MAPREDUCE-383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12729619#action_12729619 ] Hadoop QA commented on MAPREDUCE-383: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12410966/patch.HADOOP-6070 against trunk revision 792839. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/373/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/373/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/373/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/373/console This message is automatically generated. pipes combiner does not reset properly after a spill Key: MAPREDUCE-383 URL: https://issues.apache.org/jira/browse/MAPREDUCE-383 Project: Hadoop Map/Reduce Issue Type: Bug Reporter: Christian Kunz Assignee: Christian Kunz Attachments: patch.HADOOP-6070 When using a pipes combiner, the variable numBytes is not reset to 0 in spillAll, effectively reducing the effect of running a combiner to the first spill. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-710) Sqoop should read and transmit passwords in a more secure manner
[ https://issues.apache.org/jira/browse/MAPREDUCE-710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12729895#action_12729895 ] Hadoop QA commented on MAPREDUCE-710: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12413143/MAPREDUCE-710.3.patch against trunk revision 792901. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 8 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/376/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/376/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/376/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/376/console This message is automatically generated. Sqoop should read and transmit passwords in a more secure manner Key: MAPREDUCE-710 URL: https://issues.apache.org/jira/browse/MAPREDUCE-710 Project: Hadoop Map/Reduce Issue Type: Improvement Components: contrib/sqoop Reporter: Aaron Kimball Assignee: Aaron Kimball Attachments: MAPREDUCE-710.2.patch, MAPREDUCE-710.3.patch, MAPREDUCE-710.patch Sqoop's current support for passwords involves reading passwords from the command line --password foo, which makes the password visible to other users via 'ps'. An invisible-console approach should be taken. Related, Sqoop transmits passwords to mysqldump in the same fashion, which is also insecure. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-705) User-configurable quote and delimiter characters for Sqoop records and record reparsing
[ https://issues.apache.org/jira/browse/MAPREDUCE-705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12729943#action_12729943 ] Hadoop QA commented on MAPREDUCE-705: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12413046/MAPREDUCE-705.2.patch against trunk revision 793136. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 45 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. -1 release audit. The applied patch generated 316 release audit warnings (more than the trunk's current 315 warnings). -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/377/testReport/ Release audit warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/377/artifact/trunk/current/releaseAuditDiffWarnings.txt Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/377/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/377/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/377/console This message is automatically generated. User-configurable quote and delimiter characters for Sqoop records and record reparsing --- Key: MAPREDUCE-705 URL: https://issues.apache.org/jira/browse/MAPREDUCE-705 Project: Hadoop Map/Reduce Issue Type: New Feature Components: contrib/sqoop Reporter: Aaron Kimball Assignee: Aaron Kimball Attachments: MAPREDUCE-705.2.patch, MAPREDUCE-705.patch Sqoop needs a mechanism for users to govern how fields are quoted and what delimiter characters separate fields and records. With delimiters providing an unambiguous format, a parse method can reconstitute the generated record data object from a text-based representation of the same record. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-421) mapred pipes might return exit code 0 even when failing
[ https://issues.apache.org/jira/browse/MAPREDUCE-421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12729954#action_12729954 ] Hadoop QA commented on MAPREDUCE-421: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12413047/MAPREDUCE-421.patch against trunk revision 793136. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/378/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/378/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/378/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/378/console This message is automatically generated. mapred pipes might return exit code 0 even when failing --- Key: MAPREDUCE-421 URL: https://issues.apache.org/jira/browse/MAPREDUCE-421 Project: Hadoop Map/Reduce Issue Type: Bug Components: pipes Reporter: Christian Kunz Assignee: Christian Kunz Fix For: 0.20.1 Attachments: MAPREDUCE-421.patch up to hadoop 0.18.3 org.apache.hadoop.mapred.JobShell ensured that 'hadoop jar' returns non-zero exit code when the job fails. This is no longer true after moving this to org.apache.hadoop.util.RunJar. Pipes jobs submitted through cli never returned proper exit code. The main methods in org.apache.hadoop.util.RunJar. and org.apache.hadoop.mapred.pipes.Submitter should be modified to return an exit code similar to how org.apache.hadoop.mapred.JobShell did it. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-656) Change org.apache.hadoop.mapred.SequenceFile* classes to use new api
[ https://issues.apache.org/jira/browse/MAPREDUCE-656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12730363#action_12730363 ] Hadoop QA commented on MAPREDUCE-656: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12413106/patch-656.txt against trunk revision 793457. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 15 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. -1 release audit. The applied patch generated 323 release audit warnings (more than the trunk's current 315 warnings). -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/380/testReport/ Release audit warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/380/artifact/trunk/current/releaseAuditDiffWarnings.txt Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/380/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/380/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/380/console This message is automatically generated. Change org.apache.hadoop.mapred.SequenceFile* classes to use new api Key: MAPREDUCE-656 URL: https://issues.apache.org/jira/browse/MAPREDUCE-656 Project: Hadoop Map/Reduce Issue Type: Sub-task Reporter: Amareshwari Sriramadasu Assignee: Amareshwari Sriramadasu Attachments: patch-656.txt -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-739) Allow relative paths to be created inside archives.
[ https://issues.apache.org/jira/browse/MAPREDUCE-739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12730650#action_12730650 ] Hadoop QA commented on MAPREDUCE-739: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12413168/MAPREDUCE-739.patch against trunk revision 793457. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. -1 release audit. The applied patch generated 317 release audit warnings (more than the trunk's current 315 warnings). -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/382/testReport/ Release audit warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/382/artifact/trunk/current/releaseAuditDiffWarnings.txt Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/382/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/382/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/382/console This message is automatically generated. Allow relative paths to be created inside archives. --- Key: MAPREDUCE-739 URL: https://issues.apache.org/jira/browse/MAPREDUCE-739 Project: Hadoop Map/Reduce Issue Type: Improvement Components: harchive Reporter: Mahadev konar Assignee: Mahadev konar Fix For: 0.21.0 Attachments: HADOOP-3663.patch, HADOOP-3663.patch, HADOOP-3663.patch, MAPREDUCE-739.patch Archives currently stores the full path from the input sources -- since it allows multiple sources and regular expressions as inputs. So the created archives have the full path of the input sources. This is un intuitive and a user hassle. We should get rid of it and allow users to say that the created archive should be relative to some absolute path and throw an excpetion if the input does not confirm to the relative absolute path. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-750) Extensible ConnManager factory API
[ https://issues.apache.org/jira/browse/MAPREDUCE-750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12730711#action_12730711 ] Hadoop QA commented on MAPREDUCE-750: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12413182/MAPREDUCE-750.patch against trunk revision 793457. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 5 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. -1 release audit. The applied patch generated 316 release audit warnings (more than the trunk's current 315 warnings). +1 core tests. The patch passed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/383/testReport/ Release audit warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/383/artifact/trunk/current/releaseAuditDiffWarnings.txt Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/383/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/383/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/383/console This message is automatically generated. Extensible ConnManager factory API -- Key: MAPREDUCE-750 URL: https://issues.apache.org/jira/browse/MAPREDUCE-750 Project: Hadoop Map/Reduce Issue Type: Improvement Components: contrib/sqoop Reporter: Aaron Kimball Assignee: Aaron Kimball Attachments: MAPREDUCE-750.patch Sqoop uses the ConnFactory class to instantiate a ConnManager implementation based on the connect string and other arguments supplied by the user. This allows per-database logic to be encapsulated in different ConnManager instances, and dynamically chosen based on which database the user is actually importing from. But adding new ConnManager implementations requires modifying the source of a common ConnFactory class. An indirection layer should be used to delegate instantiation to a number of factory implementations which can be specified in the static configuration or at runtime. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-626) Modify TestLostTracker to improve execution time
[ https://issues.apache.org/jira/browse/MAPREDUCE-626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12731049#action_12731049 ] Hadoop QA commented on MAPREDUCE-626: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12413266/mapred-626-v2.patch against trunk revision 793457. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 6 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/386/console This message is automatically generated. Modify TestLostTracker to improve execution time Key: MAPREDUCE-626 URL: https://issues.apache.org/jira/browse/MAPREDUCE-626 Project: Hadoop Map/Reduce Issue Type: Bug Reporter: Jothi Padmanabhan Assignee: Jothi Padmanabhan Priority: Minor Attachments: mapred-626-v1.patch, mapred-626-v2.patch, mapred-626.patch, mapred-626.patch This test can be made faster with a few modifications -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-353) Allow shuffle read and connection timeouts to be configurable
[ https://issues.apache.org/jira/browse/MAPREDUCE-353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12731276#action_12731276 ] Hadoop QA commented on MAPREDUCE-353: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12413102/MR-353.v1.patch against trunk revision 794101. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/389/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/389/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/389/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/389/console This message is automatically generated. Allow shuffle read and connection timeouts to be configurable - Key: MAPREDUCE-353 URL: https://issues.apache.org/jira/browse/MAPREDUCE-353 Project: Hadoop Map/Reduce Issue Type: Improvement Affects Versions: 0.21.0 Reporter: Arun C Murthy Assignee: Ravi Gummadi Fix For: 0.21.0 Attachments: MR-353.patch, MR-353.v1.patch It would be good for latency-sensitive applications to tune the shuffle read/connection timeouts... in fact this made a huge difference to terasort since we were seeing individual shuffles stuck for upwards of 60s and had to have a very small read timeout. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-656) Change org.apache.hadoop.mapred.SequenceFile* classes to use new api
[ https://issues.apache.org/jira/browse/MAPREDUCE-656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12731338#action_12731338 ] Hadoop QA commented on MAPREDUCE-656: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12413379/patch-656-1.txt against trunk revision 794101. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 15 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. -1 release audit. The applied patch generated 323 release audit warnings (more than the trunk's current 315 warnings). +1 core tests. The patch passed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/390/testReport/ Release audit warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/390/artifact/trunk/current/releaseAuditDiffWarnings.txt Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/390/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/390/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/390/console This message is automatically generated. Change org.apache.hadoop.mapred.SequenceFile* classes to use new api Key: MAPREDUCE-656 URL: https://issues.apache.org/jira/browse/MAPREDUCE-656 Project: Hadoop Map/Reduce Issue Type: Sub-task Reporter: Amareshwari Sriramadasu Assignee: Amareshwari Sriramadasu Attachments: patch-656-1.txt, patch-656.txt -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-740) Provide summary information per job once a job is finished.
[ https://issues.apache.org/jira/browse/MAPREDUCE-740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12731345#action_12731345 ] Hadoop QA commented on MAPREDUCE-740: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12413395/MAPREDUCE-740_0_20090713_yhadoop20.patch against trunk revision 794101. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/391/console This message is automatically generated. Provide summary information per job once a job is finished. --- Key: MAPREDUCE-740 URL: https://issues.apache.org/jira/browse/MAPREDUCE-740 Project: Hadoop Map/Reduce Issue Type: New Feature Components: jobtracker Reporter: Hong Tang Assignee: Arun C Murthy Fix For: 0.21.0 Attachments: MAPREDUCE-740_0_20090709.patch, MAPREDUCE-740_0_20090713.patch, MAPREDUCE-740_0_20090713_yhadoop20.patch It would be nice if JobTracker can output a one line summary information per job once a job is finished. Otherwise, users or system administrators would end up scraping individual job history logs. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-40) Memory management variables need a backwards compatibility option after HADOOP-5881
[ https://issues.apache.org/jira/browse/MAPREDUCE-40?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12731346#action_12731346 ] Hadoop QA commented on MAPREDUCE-40: -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12413299/hadoop-5919-13-20.patch against trunk revision 794101. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 6 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/392/console This message is automatically generated. Memory management variables need a backwards compatibility option after HADOOP-5881 --- Key: MAPREDUCE-40 URL: https://issues.apache.org/jira/browse/MAPREDUCE-40 Project: Hadoop Map/Reduce Issue Type: Bug Reporter: Hemanth Yamijala Assignee: rahul k singh Priority: Blocker Attachments: hadoop-5919-1.patch, hadoop-5919-10.patch, hadoop-5919-11.patch, hadoop-5919-12-20.patch, hadoop-5919-12.patch, hadoop-5919-13-20.patch, hadoop-5919-13.patch, hadoop-5919-2.patch, hadoop-5919-3.patch, hadoop-5919-4.patch, hadoop-5919-5.patch, hadoop-5919-6.patch, hadoop-5919-7.patch, hadoop-5919-8.patch, hadoop-5919-9.patch HADOOP-5881 modified variables related to memory management without looking at the backwards compatibility angle. This JIRA is to adress the gap. Marking it a blocker for 0.20.1 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-739) Allow relative paths to be created inside archives.
[ https://issues.apache.org/jira/browse/MAPREDUCE-739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12731394#action_12731394 ] Hadoop QA commented on MAPREDUCE-739: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12413485/MAPREDUCE-739.patch against trunk revision 794101. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 34 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. -1 release audit. The applied patch generated 317 release audit warnings (more than the trunk's current 315 warnings). +1 core tests. The patch passed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/393/testReport/ Release audit warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/393/artifact/trunk/current/releaseAuditDiffWarnings.txt Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/393/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/393/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/393/console This message is automatically generated. Allow relative paths to be created inside archives. --- Key: MAPREDUCE-739 URL: https://issues.apache.org/jira/browse/MAPREDUCE-739 Project: Hadoop Map/Reduce Issue Type: Improvement Components: harchive Reporter: Mahadev konar Assignee: Mahadev konar Fix For: 0.21.0 Attachments: HADOOP-3663.patch, HADOOP-3663.patch, HADOOP-3663.patch, MAPREDUCE-739.patch, MAPREDUCE-739.patch Archives currently stores the full path from the input sources -- since it allows multiple sources and regular expressions as inputs. So the created archives have the full path of the input sources. This is un intuitive and a user hassle. We should get rid of it and allow users to say that the created archive should be relative to some absolute path and throw an excpetion if the input does not confirm to the relative absolute path. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-626) Modify TestLostTracker to improve execution time
[ https://issues.apache.org/jira/browse/MAPREDUCE-626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12731467#action_12731467 ] Hadoop QA commented on MAPREDUCE-626: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12413522/mapred-626-v3.patch against trunk revision 794223. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 6 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/394/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/394/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/394/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/394/console This message is automatically generated. Modify TestLostTracker to improve execution time Key: MAPREDUCE-626 URL: https://issues.apache.org/jira/browse/MAPREDUCE-626 Project: Hadoop Map/Reduce Issue Type: Bug Reporter: Jothi Padmanabhan Assignee: Jothi Padmanabhan Priority: Minor Attachments: mapred-626-v1.patch, mapred-626-v2.patch, mapred-626-v3.patch, mapred-626.patch, mapred-626.patch This test can be made faster with a few modifications -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-627) Modify TestTrackerBlacklistAcrossJobs to improve execution time
[ https://issues.apache.org/jira/browse/MAPREDUCE-627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12731707#action_12731707 ] Hadoop QA commented on MAPREDUCE-627: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12411630/mapred-627.patch against trunk revision 794324. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 4 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/395/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/395/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/395/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/395/console This message is automatically generated. Modify TestTrackerBlacklistAcrossJobs to improve execution time --- Key: MAPREDUCE-627 URL: https://issues.apache.org/jira/browse/MAPREDUCE-627 Project: Hadoop Map/Reduce Issue Type: Bug Reporter: Jothi Padmanabhan Assignee: Jothi Padmanabhan Priority: Minor Attachments: mapred-627.patch Some minor modifications can be made to the test case to improve test execution time -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-375) Change org.apache.hadoop.mapred.lib.NLineInputFormat and org.apache.hadoop.mapred.MapFileOutputFormat to use new api.
[ https://issues.apache.org/jira/browse/MAPREDUCE-375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12731768#action_12731768 ] Hadoop QA commented on MAPREDUCE-375: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12413533/patch-375.txt against trunk revision 794324. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 6 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. -1 release audit. The applied patch generated 316 release audit warnings (more than the trunk's current 315 warnings). -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/396/testReport/ Release audit warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/396/artifact/trunk/current/releaseAuditDiffWarnings.txt Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/396/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/396/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/396/console This message is automatically generated. Change org.apache.hadoop.mapred.lib.NLineInputFormat and org.apache.hadoop.mapred.MapFileOutputFormat to use new api. -- Key: MAPREDUCE-375 URL: https://issues.apache.org/jira/browse/MAPREDUCE-375 Project: Hadoop Map/Reduce Issue Type: Sub-task Reporter: Amareshwari Sriramadasu Assignee: Amareshwari Sriramadasu Attachments: patch-375.txt -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-705) User-configurable quote and delimiter characters for Sqoop records and record reparsing
[ https://issues.apache.org/jira/browse/MAPREDUCE-705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12731903#action_12731903 ] Hadoop QA commented on MAPREDUCE-705: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12413589/MAPREDUCE-705.3.patch against trunk revision 794541. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 48 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. -1 release audit. The applied patch generated 316 release audit warnings (more than the trunk's current 315 warnings). -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/399/testReport/ Release audit warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/399/artifact/trunk/current/releaseAuditDiffWarnings.txt Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/399/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/399/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/399/console This message is automatically generated. User-configurable quote and delimiter characters for Sqoop records and record reparsing --- Key: MAPREDUCE-705 URL: https://issues.apache.org/jira/browse/MAPREDUCE-705 Project: Hadoop Map/Reduce Issue Type: New Feature Components: contrib/sqoop Reporter: Aaron Kimball Assignee: Aaron Kimball Attachments: MAPREDUCE-705.2.patch, MAPREDUCE-705.3.patch, MAPREDUCE-705.patch Sqoop needs a mechanism for users to govern how fields are quoted and what delimiter characters separate fields and records. With delimiters providing an unambiguous format, a parse method can reconstitute the generated record data object from a text-based representation of the same record. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-764) TypedBytesInput's readRaw() does not preserve custom type codes
[
https://issues.apache.org/jira/browse/MAPREDUCE-764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12732183#action_12732183
]
Hadoop QA commented on MAPREDUCE-764:
-
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12413658/MAPREDUCE-764.patch
against trunk revision 794637.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac
compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of
release audit warnings.
-1 core tests. The patch failed core unit tests.
-1 contrib tests. The patch failed contrib unit tests.
Test results:
http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/403/testReport/
Findbugs warnings:
http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/403/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results:
http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/403/artifact/trunk/build/test/checkstyle-errors.html
Console output:
http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/403/console
This message is automatically generated.
TypedBytesInput's readRaw() does not preserve custom type codes
---
Key: MAPREDUCE-764
URL: https://issues.apache.org/jira/browse/MAPREDUCE-764
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: contrib/streaming
Affects Versions: 0.21.0
Reporter: Klaas Bosteels
Assignee: Klaas Bosteels
Attachments: MAPREDUCE-764.patch, MAPREDUCE-764.patch
The typed bytes format supports byte sequences of the form {{custom type
code length bytes}}. When reading such a sequence via
{{TypedBytesInput}}'s {{readRaw()}} method, however, the returned sequence
currently is {{0 length bytes}} (0 is the type code for a bytes array),
which leads to bugs such as the one described
[here|http://dumbo.assembla.com/spaces/dumbo/tickets/54].
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-369) Change org.apache.hadoop.mapred.lib.MultipleInputs to use new api.
[ https://issues.apache.org/jira/browse/MAPREDUCE-369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12732286#action_12732286 ] Hadoop QA commented on MAPREDUCE-369: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12413659/patch-369.txt against trunk revision 794637. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 6 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/404/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/404/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/404/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/404/console This message is automatically generated. Change org.apache.hadoop.mapred.lib.MultipleInputs to use new api. -- Key: MAPREDUCE-369 URL: https://issues.apache.org/jira/browse/MAPREDUCE-369 Project: Hadoop Map/Reduce Issue Type: Sub-task Reporter: Amareshwari Sriramadasu Assignee: Amareshwari Sriramadasu Attachments: patch-369.txt -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-765) eliminate the usage of FileSystem.create( ) depracated by Hadoop-5438
[ https://issues.apache.org/jira/browse/MAPREDUCE-765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12732346#action_12732346 ] Hadoop QA commented on MAPREDUCE-765: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12413550/mapreduce-765-2009-07-15.patch against trunk revision 794942. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/405/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/405/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/405/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/405/console This message is automatically generated. eliminate the usage of FileSystem.create( ) depracated by Hadoop-5438 -- Key: MAPREDUCE-765 URL: https://issues.apache.org/jira/browse/MAPREDUCE-765 Project: Hadoop Map/Reduce Issue Type: Improvement Reporter: He Yongqiang Priority: Minor Attachments: mapreduce-765-2009-07-15.patch -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-548) Global scheduling in the Fair Scheduler
[ https://issues.apache.org/jira/browse/MAPREDUCE-548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12732826#action_12732826 ] Hadoop QA commented on MAPREDUCE-548: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12413886/mapreduce-548-v3.patch against trunk revision 794942. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/408/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/408/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/408/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/408/console This message is automatically generated. Global scheduling in the Fair Scheduler --- Key: MAPREDUCE-548 URL: https://issues.apache.org/jira/browse/MAPREDUCE-548 Project: Hadoop Map/Reduce Issue Type: New Feature Reporter: Matei Zaharia Assignee: Matei Zaharia Fix For: 0.21.0 Attachments: fs-global-v0.patch, hadoop-4667-v1.patch, hadoop-4667-v1b.patch, hadoop-4667-v2.patch, HADOOP-4667_api.patch, mapreduce-548-v1.patch, mapreduce-548-v2.patch, mapreduce-548-v3.patch, mapreduce-548.patch The current schedulers in Hadoop all examine a single job on every heartbeat when choosing which tasks to assign, choosing the job based on FIFO or fair sharing. There are inherent limitations to this approach. For example, if the job at the front of the queue is small (e.g. 10 maps, in a cluster of 100 nodes), then on average it will launch only one local map on the first 10 heartbeats while it is at the head of the queue. This leads to very poor locality for small jobs. Instead, we need a more global view of scheduling that can look at multiple jobs. To resolve the locality problem, we will use the following algorithm: - If the job at the head of the queue has no node-local task to launch, skip it and look through other jobs. - If a job has waited at least T1 seconds while being skipped, also allow it to launch rack-local tasks. - If a job has waited at least T2 T1 seconds, also allow it to launch off-rack tasks. This algorithm improves locality while bounding the delay that any job experiences in launching a task. It turns out that whether waiting is useful depends on how many tasks are left in the job - the probability of getting a heartbeat from a node with a local task - and on whether the job is CPU or IO bound. Thus there may be logic for removing the wait on the last few tasks in the job. As a related issue, once we allow global scheduling, we can launch multiple tasks per heartbeat, as in HADOOP-3136. The initial implementation of HADOOP-3136 adversely affected performance because it only launched multiple tasks from the same job, but with the wait rule above, we will only do this for jobs that are allowed to launch non-local tasks. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-765) eliminate the usage of FileSystem.create( ) depracated by Hadoop-5438
[ https://issues.apache.org/jira/browse/MAPREDUCE-765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12732876#action_12732876 ] Hadoop QA commented on MAPREDUCE-765: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12413908/mapreduce-765-2009-07-18.patch against trunk revision 794942. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/409/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/409/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/409/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/409/console This message is automatically generated. eliminate the usage of FileSystem.create( ) depracated by Hadoop-5438 -- Key: MAPREDUCE-765 URL: https://issues.apache.org/jira/browse/MAPREDUCE-765 Project: Hadoop Map/Reduce Issue Type: Improvement Reporter: He Yongqiang Priority: Minor Attachments: mapreduce-765-2009-07-15.patch, mapreduce-765-2009-07-18.patch -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-712) RandomTextWriter example is CPU bound
[ https://issues.apache.org/jira/browse/MAPREDUCE-712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12732962#action_12732962 ] Hadoop QA commented on MAPREDUCE-712: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12413936/MR712-1.patch against trunk revision 795454. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/410/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/410/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/410/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/410/console This message is automatically generated. RandomTextWriter example is CPU bound - Key: MAPREDUCE-712 URL: https://issues.apache.org/jira/browse/MAPREDUCE-712 Project: Hadoop Map/Reduce Issue Type: Improvement Components: examples Affects Versions: 0.20.1, 0.21.0 Environment: ~200 nodes cluster Each node has the following configuration: Processors: 2 x Xeon L5420 2.50GHz (8 cores) - Harpertown C0, 64-bit, quad-core (8 CPUs) 4 Disks 16 GB RAM Linux 2.6 Hadoop version: trunk Reporter: Khaled Elmeleegy Assignee: Chris Douglas Attachments: MR712-0.patch, MR712-1.patch Running the RandomTextWritter example job ( from the examples jar) pegs the machiens' CPUs. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-740) Provide summary information per job once a job is finished.
[ https://issues.apache.org/jira/browse/MAPREDUCE-740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12732969#action_12732969 ] Hadoop QA commented on MAPREDUCE-740: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12413941/MAPREDUCE-740_2_20090717_yhadoop20.patch against trunk revision 795470. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/411/console This message is automatically generated. Provide summary information per job once a job is finished. --- Key: MAPREDUCE-740 URL: https://issues.apache.org/jira/browse/MAPREDUCE-740 Project: Hadoop Map/Reduce Issue Type: New Feature Components: jobtracker Reporter: Hong Tang Assignee: Arun C Murthy Fix For: 0.21.0 Attachments: MAPREDUCE-740_0_20090709.patch, MAPREDUCE-740_0_20090713.patch, MAPREDUCE-740_0_20090713_yhadoop20.patch, MAPREDUCE-740_1_20090716.patch, MAPREDUCE-740_1_20090716_yhadoop20.patch, MAPREDUCE-740_2_20090717.patch, MAPREDUCE-740_2_20090717_yhadoop20.patch It would be nice if JobTracker can output a one line summary information per job once a job is finished. Otherwise, users or system administrators would end up scraping individual job history logs. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-789) Oracle support for Sqoop
[ https://issues.apache.org/jira/browse/MAPREDUCE-789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12735883#action_12735883 ] Hadoop QA commented on MAPREDUCE-789: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12414235/MAPREDUCE-789.patch against trunk revision 798239. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 5 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/425/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/425/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/425/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/425/console This message is automatically generated. Oracle support for Sqoop Key: MAPREDUCE-789 URL: https://issues.apache.org/jira/browse/MAPREDUCE-789 Project: Hadoop Map/Reduce Issue Type: Improvement Components: contrib/sqoop Reporter: Aaron Kimball Assignee: Aaron Kimball Attachments: MAPREDUCE-789.patch A separate ConnManager is needed for Oracle to support its slightly different syntax and configuration -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-706) Support for FIFO pools in the fair scheduler
[ https://issues.apache.org/jira/browse/MAPREDUCE-706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12737034#action_12737034 ] Hadoop QA commented on MAPREDUCE-706: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12414820/mapreduce-706.v1.patch against trunk revision 799126. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 9 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. -1 release audit. The applied patch generated 210 release audit warnings (more than the trunk's current 203 warnings). +1 core tests. The patch passed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/427/testReport/ Release audit warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/427/artifact/trunk/patchprocess/releaseAuditDiffWarnings.txt Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/427/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/427/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/427/console This message is automatically generated. Support for FIFO pools in the fair scheduler Key: MAPREDUCE-706 URL: https://issues.apache.org/jira/browse/MAPREDUCE-706 Project: Hadoop Map/Reduce Issue Type: New Feature Components: contrib/fair-share Reporter: Matei Zaharia Assignee: Matei Zaharia Attachments: fsdesigndoc.pdf, fsdesigndoc.tex, mapreduce-706.patch, mapreduce-706.v1.patch The fair scheduler should support making the internal scheduling algorithm for some pools be FIFO instead of fair sharing in order to work better for batch workloads. FIFO pools will behave exactly like the current default scheduler, sorting jobs by priority and then submission time. Pools will have their scheduling algorithm set through the pools config file, and it will be changeable at runtime. To support this feature, I'm also changing the internal logic of the fair scheduler to no longer use deficits. Instead, for fair sharing, we will assign tasks to the job farthest below its share as a ratio of its share. This is easier to combine with other scheduling algorithms and leads to a more stable sharing situation, avoiding unfairness issues brought up in MAPREDUCE-543 and MAPREDUCE-544 that happen when some jobs have long tasks. The new preemption (MAPREDUCE-551) will ensure that critical jobs can gain their fair share within a bounded amount of time. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-792) javac warnings in DBInputFormat
[ https://issues.apache.org/jira/browse/MAPREDUCE-792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12737069#action_12737069 ] Hadoop QA commented on MAPREDUCE-792: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12414303/MAPREDUCE-792.2.patch against trunk revision 799126. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/428/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/428/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/428/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/428/console This message is automatically generated. javac warnings in DBInputFormat --- Key: MAPREDUCE-792 URL: https://issues.apache.org/jira/browse/MAPREDUCE-792 Project: Hadoop Map/Reduce Issue Type: Bug Reporter: Aaron Kimball Assignee: Aaron Kimball Priority: Minor Attachments: MAPREDUCE-792.2.patch, MAPREDUCE-792.patch MAPREDUCE-716 introduces javac warnings -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-797) MRUnit MapReduceDriver should support combiners
[ https://issues.apache.org/jira/browse/MAPREDUCE-797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12737138#action_12737138 ] Hadoop QA commented on MAPREDUCE-797: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12414376/MAPREDUCE-797.patch against trunk revision 799126. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/429/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/429/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/429/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/429/console This message is automatically generated. MRUnit MapReduceDriver should support combiners --- Key: MAPREDUCE-797 URL: https://issues.apache.org/jira/browse/MAPREDUCE-797 Project: Hadoop Map/Reduce Issue Type: Improvement Reporter: Aaron Kimball Assignee: Aaron Kimball Attachments: MAPREDUCE-797.patch The MapReduceDriver allows you to specify a mapper and a reducer class with a simple sort/shuffle between the passes. It would be nice to also support another Reducer implementation being used as a combiner in the middle. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-670) Create target for 10 minute patch test build for mapreduce
[ https://issues.apache.org/jira/browse/MAPREDUCE-670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12738236#action_12738236 ] Hadoop QA commented on MAPREDUCE-670: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12415093/mapred-670-v2.patch against trunk revision 799551. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 48 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/432/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/432/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/432/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/432/console This message is automatically generated. Create target for 10 minute patch test build for mapreduce --- Key: MAPREDUCE-670 URL: https://issues.apache.org/jira/browse/MAPREDUCE-670 Project: Hadoop Map/Reduce Issue Type: Test Components: build Reporter: Jothi Padmanabhan Assignee: Jothi Padmanabhan Attachments: FastTestsInfo.xls, mapred-670-v1.patch, mapred-670-v2.patch, mapred-670.patch Creating a new Jira to track HADOOP-5628 for MapReduce -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-800) MRUnit should support the new API
[ https://issues.apache.org/jira/browse/MAPREDUCE-800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12738240#action_12738240 ] Hadoop QA commented on MAPREDUCE-800: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12414379/MAPREDUCE-800.patch against trunk revision 800232. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 14 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/433/console This message is automatically generated. MRUnit should support the new API - Key: MAPREDUCE-800 URL: https://issues.apache.org/jira/browse/MAPREDUCE-800 Project: Hadoop Map/Reduce Issue Type: New Feature Reporter: Aaron Kimball Assignee: Aaron Kimball Attachments: MAPREDUCE-800.patch MRUnit's TestDriver implementations use the old org.apache.hadoop.mapred-based classes. TestDrivers and associated mock object implementations are required for org.apache.hadoop.mapreduce-based code. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-813) Streaming Doc and M/R-Tutorial Doc - updates
[ https://issues.apache.org/jira/browse/MAPREDUCE-813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12738345#action_12738345 ] Hadoop QA commented on MAPREDUCE-813: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12415063/MAPREDUCE-813-2.patch against trunk revision 800329. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 6 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/438/console This message is automatically generated. Streaming Doc and M/R-Tutorial Doc - updates - Key: MAPREDUCE-813 URL: https://issues.apache.org/jira/browse/MAPREDUCE-813 Project: Hadoop Map/Reduce Issue Type: Task Components: documentation Affects Versions: 0.21.0 Reporter: Corinne Chandel Priority: Minor Fix For: 0.21.0 Attachments: mapred_tutorial.pdf, MAPREDUCE-813-2.patch, MAPREDUCE-813.patch, streaming.pdf This JIRA addresses issues in the Streaming doc that also require a cross-link to and update in the M/R Tutorial doc. All changes approved by the reviewing engineer. Streaming issues: 1. During the execution of a streaming job, the names of the mapred parameters are transformed. The dots ( . ) become underscores ( _ ). Docs affected: streaming and m/r tutorial (new sub-sections added under Task Execution Environment section) 2. For -files and -archives options, Hadoop now creates symlink with same name as file (user-defined symlinks, #mysymlink, currently not supported) Docs affected:streaming 3. Streaming supports streaming command options and generic command options. Generic options must be placed before streaming options, otherwise command fails. Docs affected: streaming (reorganized the streaming doc to make distinctions between 2 sets of command options more clear) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-372) Change org.apache.hadoop.mapred.lib.ChainMapper/Reducer to use new api.
[ https://issues.apache.org/jira/browse/MAPREDUCE-372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12738354#action_12738354 ] Hadoop QA commented on MAPREDUCE-372: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12414890/patch-372.txt against trunk revision 800329. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/439/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/439/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/439/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/439/console This message is automatically generated. Change org.apache.hadoop.mapred.lib.ChainMapper/Reducer to use new api. --- Key: MAPREDUCE-372 URL: https://issues.apache.org/jira/browse/MAPREDUCE-372 Project: Hadoop Map/Reduce Issue Type: Sub-task Reporter: Amareshwari Sriramadasu Assignee: Amareshwari Sriramadasu Fix For: 0.21.0 Attachments: patch-372.txt -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-372) Change org.apache.hadoop.mapred.lib.ChainMapper/Reducer to use new api.
[ https://issues.apache.org/jira/browse/MAPREDUCE-372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12738923#action_12738923 ] Hadoop QA commented on MAPREDUCE-372: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12414890/patch-372.txt against trunk revision 800684. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/440/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/440/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/440/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/440/console This message is automatically generated. Change org.apache.hadoop.mapred.lib.ChainMapper/Reducer to use new api. --- Key: MAPREDUCE-372 URL: https://issues.apache.org/jira/browse/MAPREDUCE-372 Project: Hadoop Map/Reduce Issue Type: Sub-task Reporter: Amareshwari Sriramadasu Assignee: Amareshwari Sriramadasu Fix For: 0.21.0 Attachments: patch-372.txt -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-706) Support for FIFO pools in the fair scheduler
[ https://issues.apache.org/jira/browse/MAPREDUCE-706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12739939#action_12739939 ] Hadoop QA commented on MAPREDUCE-706: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12415552/mapreduce-706.v3.patch against trunk revision 800693. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 9 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/443/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/443/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/443/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/443/console This message is automatically generated. Support for FIFO pools in the fair scheduler Key: MAPREDUCE-706 URL: https://issues.apache.org/jira/browse/MAPREDUCE-706 Project: Hadoop Map/Reduce Issue Type: New Feature Components: contrib/fair-share Reporter: Matei Zaharia Assignee: Matei Zaharia Attachments: fsdesigndoc.pdf, fsdesigndoc.tex, mapreduce-706.patch, mapreduce-706.v1.patch, mapreduce-706.v2.patch, mapreduce-706.v3.patch The fair scheduler should support making the internal scheduling algorithm for some pools be FIFO instead of fair sharing in order to work better for batch workloads. FIFO pools will behave exactly like the current default scheduler, sorting jobs by priority and then submission time. Pools will have their scheduling algorithm set through the pools config file, and it will be changeable at runtime. To support this feature, I'm also changing the internal logic of the fair scheduler to no longer use deficits. Instead, for fair sharing, we will assign tasks to the job farthest below its share as a ratio of its share. This is easier to combine with other scheduling algorithms and leads to a more stable sharing situation, avoiding unfairness issues brought up in MAPREDUCE-543 and MAPREDUCE-544 that happen when some jobs have long tasks. The new preemption (MAPREDUCE-551) will ensure that critical jobs can gain their fair share within a bounded amount of time. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-749) Make Sqoop unit tests more Hudson-friendly
[ https://issues.apache.org/jira/browse/MAPREDUCE-749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12740344#action_12740344 ] Hadoop QA commented on MAPREDUCE-749: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12415424/MAPREDUCE-749.3.patch against trunk revision 801517. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 5 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/447/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/447/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/447/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/447/console This message is automatically generated. Make Sqoop unit tests more Hudson-friendly -- Key: MAPREDUCE-749 URL: https://issues.apache.org/jira/browse/MAPREDUCE-749 Project: Hadoop Map/Reduce Issue Type: Improvement Components: contrib/sqoop Reporter: Aaron Kimball Assignee: Aaron Kimball Attachments: MAPREDUCE-749.2.patch, MAPREDUCE-749.3.patch, MAPREDUCE-749.patch Hudson servers (other than Apache's) need to be able to run the sqoop unit tests which depend on thirdparty JDBC drivers / database implementations. The build.xml needs some refactoring to make this happen. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-375) Change org.apache.hadoop.mapred.lib.NLineInputFormat and org.apache.hadoop.mapred.MapFileOutputFormat to use new api.
[ https://issues.apache.org/jira/browse/MAPREDUCE-375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12740409#action_12740409 ] Hadoop QA commented on MAPREDUCE-375: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12415117/patch-375-2.txt against trunk revision 801517. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 6 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/448/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/448/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/448/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/448/console This message is automatically generated. Change org.apache.hadoop.mapred.lib.NLineInputFormat and org.apache.hadoop.mapred.MapFileOutputFormat to use new api. -- Key: MAPREDUCE-375 URL: https://issues.apache.org/jira/browse/MAPREDUCE-375 Project: Hadoop Map/Reduce Issue Type: Sub-task Reporter: Amareshwari Sriramadasu Assignee: Amareshwari Sriramadasu Fix For: 0.21.0 Attachments: patch-375-1.txt, patch-375-2.txt, patch-375.txt -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-750) Extensible ConnManager factory API
[ https://issues.apache.org/jira/browse/MAPREDUCE-750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12740511#action_12740511 ] Hadoop QA commented on MAPREDUCE-750: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12415690/MAPREDUCE-750.2.patch against trunk revision 801517. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 5 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/451/console This message is automatically generated. Extensible ConnManager factory API -- Key: MAPREDUCE-750 URL: https://issues.apache.org/jira/browse/MAPREDUCE-750 Project: Hadoop Map/Reduce Issue Type: Improvement Components: contrib/sqoop Reporter: Aaron Kimball Assignee: Aaron Kimball Attachments: MAPREDUCE-750.2.patch, MAPREDUCE-750.patch Sqoop uses the ConnFactory class to instantiate a ConnManager implementation based on the connect string and other arguments supplied by the user. This allows per-database logic to be encapsulated in different ConnManager instances, and dynamically chosen based on which database the user is actually importing from. But adding new ConnManager implementations requires modifying the source of a common ConnFactory class. An indirection layer should be used to delegate instantiation to a number of factory implementations which can be specified in the static configuration or at runtime. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-478) separate jvm param for mapper and reducer
[ https://issues.apache.org/jira/browse/MAPREDUCE-478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12740513#action_12740513 ] Hadoop QA commented on MAPREDUCE-478: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12415805/MAPREDUCE-478_1_20090806_yhadoop20.patch against trunk revision 801954. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 19 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/452/console This message is automatically generated. separate jvm param for mapper and reducer - Key: MAPREDUCE-478 URL: https://issues.apache.org/jira/browse/MAPREDUCE-478 Project: Hadoop Map/Reduce Issue Type: Improvement Reporter: Koji Noguchi Assignee: Arun C Murthy Priority: Minor Fix For: 0.21.0 Attachments: HADOOP-5684_0_20090420.patch, MAPREDUCE-478_0_20090804.patch, MAPREDUCE-478_0_20090804_yhadoop20.patch, MAPREDUCE-478_1_20090806.patch, MAPREDUCE-478_1_20090806_yhadoop20.patch Memory footprint of mapper and reducer can differ. It would be nice if we can pass different jvm param (mapred.child.java.opts) for mappers and reducers. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-479) Add reduce ID to shuffle clienttrace
[ https://issues.apache.org/jira/browse/MAPREDUCE-479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12740584#action_12740584 ] Hadoop QA commented on MAPREDUCE-479: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12415747/MAPREDUCE-479-4.patch against trunk revision 801959. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/453/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/453/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/453/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/453/console This message is automatically generated. Add reduce ID to shuffle clienttrace Key: MAPREDUCE-479 URL: https://issues.apache.org/jira/browse/MAPREDUCE-479 Project: Hadoop Map/Reduce Issue Type: Improvement Affects Versions: 0.21.0 Reporter: Jiaqi Tan Assignee: Jiaqi Tan Priority: Minor Fix For: 0.21.0 Attachments: HADOOP-6013.patch, MAPREDUCE-479-1.patch, MAPREDUCE-479-2.patch, MAPREDUCE-479-3.patch, MAPREDUCE-479-4.patch, MAPREDUCE-479.patch Current clienttrace messages from shuffles note only the destination map ID but not the source reduce ID. Having both source and destination ID of each shuffle enables full tracing of execution. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-825) JobClient completion poll interval of 5s causes slow tests in local mode
[ https://issues.apache.org/jira/browse/MAPREDUCE-825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12740726#action_12740726 ] Hadoop QA commented on MAPREDUCE-825: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12415772/MAPREDUCE-825.2.patch against trunk revision 801959. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/454/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/454/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/454/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/454/console This message is automatically generated. JobClient completion poll interval of 5s causes slow tests in local mode Key: MAPREDUCE-825 URL: https://issues.apache.org/jira/browse/MAPREDUCE-825 Project: Hadoop Map/Reduce Issue Type: Bug Reporter: Aaron Kimball Assignee: Aaron Kimball Priority: Minor Attachments: completion-poll-interval.patch, MAPREDUCE-825.2.patch The JobClient.NetworkedJob.waitForCompletion() method polls for job completion every 5 seconds. When running a set of short tests in pseudo-distributed mode, this is unnecessarily slow and causes lots of wasted time. When bandwidth is not scarce, setting the poll interval to 100 ms results in a 4x speedup in some tests. This interval should be parametrized to allow users to control the interval for testing purposes. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-799) Some of MRUnit's self-tests were not being run
[ https://issues.apache.org/jira/browse/MAPREDUCE-799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12740808#action_12740808 ] Hadoop QA commented on MAPREDUCE-799: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12414378/MAPREDUCE-799.patch against trunk revision 801959. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 9 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/455/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/455/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/455/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/455/console This message is automatically generated. Some of MRUnit's self-tests were not being run -- Key: MAPREDUCE-799 URL: https://issues.apache.org/jira/browse/MAPREDUCE-799 Project: Hadoop Map/Reduce Issue Type: Bug Reporter: Aaron Kimball Assignee: Aaron Kimball Attachments: MAPREDUCE-799.patch Due to method naming issues, some test cases were not being executed. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-706) Support for FIFO pools in the fair scheduler
[ https://issues.apache.org/jira/browse/MAPREDUCE-706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12740847#action_12740847 ] Hadoop QA commented on MAPREDUCE-706: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12415798/mapreduce-706.v4.patch against trunk revision 801959. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 9 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/456/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/456/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/456/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/456/console This message is automatically generated. Support for FIFO pools in the fair scheduler Key: MAPREDUCE-706 URL: https://issues.apache.org/jira/browse/MAPREDUCE-706 Project: Hadoop Map/Reduce Issue Type: New Feature Components: contrib/fair-share Reporter: Matei Zaharia Assignee: Matei Zaharia Attachments: fsdesigndoc.pdf, fsdesigndoc.tex, mapreduce-706.patch, mapreduce-706.v1.patch, mapreduce-706.v2.patch, mapreduce-706.v3.patch, mapreduce-706.v4.patch The fair scheduler should support making the internal scheduling algorithm for some pools be FIFO instead of fair sharing in order to work better for batch workloads. FIFO pools will behave exactly like the current default scheduler, sorting jobs by priority and then submission time. Pools will have their scheduling algorithm set through the pools config file, and it will be changeable at runtime. To support this feature, I'm also changing the internal logic of the fair scheduler to no longer use deficits. Instead, for fair sharing, we will assign tasks to the job farthest below its share as a ratio of its share. This is easier to combine with other scheduling algorithms and leads to a more stable sharing situation, avoiding unfairness issues brought up in MAPREDUCE-543 and MAPREDUCE-544 that happen when some jobs have long tasks. The new preemption (MAPREDUCE-551) will ensure that critical jobs can gain their fair share within a bounded amount of time. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-779) Add node health failures into JobTrackerStatistics
[
https://issues.apache.org/jira/browse/MAPREDUCE-779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12740875#action_12740875
]
Hadoop QA commented on MAPREDUCE-779:
-
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12415816/mapreduce-779-4.patch
against trunk revision 801959.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 10 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac
compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of
release audit warnings.
+1 core tests. The patch passed core unit tests.
-1 contrib tests. The patch failed contrib unit tests.
Test results:
http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/457/testReport/
Findbugs warnings:
http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/457/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results:
http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/457/artifact/trunk/build/test/checkstyle-errors.html
Console output:
http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/457/console
This message is automatically generated.
Add node health failures into JobTrackerStatistics
--
Key: MAPREDUCE-779
URL: https://issues.apache.org/jira/browse/MAPREDUCE-779
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: jobtracker
Reporter: Sreekanth Ramakrishnan
Assignee: Sreekanth Ramakrishnan
Attachments: mapreduce-779-1.patch, mapreduce-779-2.patch,
mapreduce-779-3.patch, mapreduce-779-4.patch
Add the node health failure counts into {{JobTrackerStatistics}}.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-840) DBInputFormat leaves open transaction
[ https://issues.apache.org/jira/browse/MAPREDUCE-840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12740902#action_12740902 ] Hadoop QA commented on MAPREDUCE-840: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12415893/MAPREDUCE-840.patch against trunk revision 801959. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/458/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/458/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/458/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/458/console This message is automatically generated. DBInputFormat leaves open transaction - Key: MAPREDUCE-840 URL: https://issues.apache.org/jira/browse/MAPREDUCE-840 Project: Hadoop Map/Reduce Issue Type: Bug Reporter: Aaron Kimball Assignee: Aaron Kimball Priority: Minor Attachments: MAPREDUCE-840.patch DBInputFormat.getSplits() does not connection.commit() after the COUNT query. This can leave an open transaction against the database which interferes with other connections to the same table. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-750) Extensible ConnManager factory API
[ https://issues.apache.org/jira/browse/MAPREDUCE-750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12740998#action_12740998 ] Hadoop QA commented on MAPREDUCE-750: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12415903/MAPREDUCE-750.3.patch against trunk revision 801959. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 5 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. -1 release audit. The applied patch generated 204 release audit warnings (more than the trunk's current 203 warnings). +1 core tests. The patch passed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/459/testReport/ Release audit warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/459/artifact/trunk/patchprocess/releaseAuditDiffWarnings.txt Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/459/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/459/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/459/console This message is automatically generated. Extensible ConnManager factory API -- Key: MAPREDUCE-750 URL: https://issues.apache.org/jira/browse/MAPREDUCE-750 Project: Hadoop Map/Reduce Issue Type: Improvement Components: contrib/sqoop Reporter: Aaron Kimball Assignee: Aaron Kimball Attachments: MAPREDUCE-750.2.patch, MAPREDUCE-750.3.patch, MAPREDUCE-750.patch Sqoop uses the ConnFactory class to instantiate a ConnManager implementation based on the connect string and other arguments supplied by the user. This allows per-database logic to be encapsulated in different ConnManager instances, and dynamically chosen based on which database the user is actually importing from. But adding new ConnManager implementations requires modifying the source of a common ConnFactory class. An indirection layer should be used to delegate instantiation to a number of factory implementations which can be specified in the static configuration or at runtime. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-838) Task succeeds even when committer.commitTask fails with IOException
[
https://issues.apache.org/jira/browse/MAPREDUCE-838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12741243#action_12741243
]
Hadoop QA commented on MAPREDUCE-838:
-
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12416024/patch-838.txt
against trunk revision 802645.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac
compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of
release audit warnings.
-1 core tests. The patch failed core unit tests.
-1 contrib tests. The patch failed contrib unit tests.
Test results:
http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/460/testReport/
Findbugs warnings:
http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/460/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results:
http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/460/artifact/trunk/build/test/checkstyle-errors.html
Console output:
http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/460/console
This message is automatically generated.
Task succeeds even when committer.commitTask fails with IOException
---
Key: MAPREDUCE-838
URL: https://issues.apache.org/jira/browse/MAPREDUCE-838
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: task
Affects Versions: 0.20.1
Reporter: Koji Noguchi
Assignee: Amareshwari Sriramadasu
Priority: Blocker
Fix For: 0.20.1
Attachments: patch-838-0.20.txt, patch-838.txt
In MAPREDUCE-837, job succeeded with empty output even though all the tasks
were throwing IOException at commiter.commitTask.
{noformat}
2009-08-07 17:51:47,458 INFO org.apache.hadoop.mapred.TaskRunner: Task
attempt_200907301448_8771_r_00_0 is allowed to commit now
2009-08-07 17:51:47,466 WARN org.apache.hadoop.mapred.TaskRunner: Failure
committing: java.io.IOException: Can not get the relative path: \
base =
hdfs://mynamenode:8020/user/knoguchi/test2.har/_temporary/_attempt_200907301448_8771_r_00_0
\
child =
hdfs://mynamenode/user/knoguchi/test2.har/_temporary/_attempt_200907301448_8771_r_00_0/_index
at
org.apache.hadoop.mapred.FileOutputCommitter.getFinalPath(FileOutputCommitter.java:150)
at
org.apache.hadoop.mapred.FileOutputCommitter.moveTaskOutputs(FileOutputCommitter.java:106)
at
org.apache.hadoop.mapred.FileOutputCommitter.moveTaskOutputs(FileOutputCommitter.java:126)
at
org.apache.hadoop.mapred.FileOutputCommitter.commitTask(FileOutputCommitter.java:86)
at
org.apache.hadoop.mapred.OutputCommitter.commitTask(OutputCommitter.java:171)
at org.apache.hadoop.mapred.Task.commit(Task.java:768)
at org.apache.hadoop.mapred.Task.done(Task.java:692)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:417)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
2009-08-07 17:51:47,468 WARN org.apache.hadoop.mapred.TaskRunner: Failure
asking whether task can commit: java.io.IOException: \
Can not get the relative path: base =
hdfs://mynamenode:8020/user/knoguchi/test2.har/_temporary/_attempt_200907301448_8771_r_00_0
\
child =
hdfs://mynamenode/user/knoguchi/test2.har/_temporary/_attempt_200907301448_8771_r_00_0/_index
at
org.apache.hadoop.mapred.FileOutputCommitter.getFinalPath(FileOutputCommitter.java:150)
at
org.apache.hadoop.mapred.FileOutputCommitter.moveTaskOutputs(FileOutputCommitter.java:106)
at
org.apache.hadoop.mapred.FileOutputCommitter.moveTaskOutputs(FileOutputCommitter.java:126)
at
org.apache.hadoop.mapred.FileOutputCommitter.commitTask(FileOutputCommitter.java:86)
at
org.apache.hadoop.mapred.OutputCommitter.commitTask(OutputCommitter.java:171)
at org.apache.hadoop.mapred.Task.commit(Task.java:768)
at org.apache.hadoop.mapred.Task.done(Task.java:692)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:417)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
2009-08-07 17:51:47,469 INFO org.apache.hadoop.mapred.TaskRunner: Task
attempt_200907301448_8771_r_00_0 is allowed to commit now
2009-08-07 17:51:47,472 INFO org.apache.hadoop.mapred.TaskRunner: Task
'attempt_200907301448_8771_r_00_0' done.
{noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the
[jira] Commented: (MAPREDUCE-777) A method for finding and tracking jobs from the new API
[ https://issues.apache.org/jira/browse/MAPREDUCE-777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12741326#action_12741326 ] Hadoop QA commented on MAPREDUCE-777: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12416042/patch-777.txt against trunk revision 802645. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 9 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. -1 javac. The applied patch generated 2272 javac compiler warnings (more than the trunk's current 2232 warnings). +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/461/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/461/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/461/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/461/console This message is automatically generated. A method for finding and tracking jobs from the new API --- Key: MAPREDUCE-777 URL: https://issues.apache.org/jira/browse/MAPREDUCE-777 Project: Hadoop Map/Reduce Issue Type: New Feature Components: client Reporter: Owen O'Malley Assignee: Amareshwari Sriramadasu Fix For: 0.21.0 Attachments: patch-777.txt We need to create a replacement interface for the JobClient API in the new interface. In particular, the user needs to be able to query and track jobs that were launched by other processes. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-842) Per-job local data on the TaskTracker node should have right access-control
[ https://issues.apache.org/jira/browse/MAPREDUCE-842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12741664#action_12741664 ] Hadoop QA commented on MAPREDUCE-842: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12416038/HADOOP-4491-20090810.3.txt against trunk revision 802954. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 47 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 5 new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/463/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/463/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/463/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/463/console This message is automatically generated. Per-job local data on the TaskTracker node should have right access-control --- Key: MAPREDUCE-842 URL: https://issues.apache.org/jira/browse/MAPREDUCE-842 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: tasktracker Reporter: Arun C Murthy Assignee: Vinod K V Attachments: HADOOP-4491-20090623-common.1.txt, HADOOP-4491-20090623-mapred.1.txt, HADOOP-4491-20090703-common.1.txt, HADOOP-4491-20090703-common.txt, HADOOP-4491-20090703.1.txt, HADOOP-4491-20090703.txt, HADOOP-4491-20090707-common.txt, HADOOP-4491-20090707.txt, HADOOP-4491-20090716-mapred.txt, HADOOP-4491-20090803.1.txt, HADOOP-4491-20090803.txt, HADOOP-4491-20090806.txt, HADOOP-4491-20090807.2.txt, HADOOP-4491-20090810.1.txt, HADOOP-4491-20090810.3.txt -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-842) Per-job local data on the TaskTracker node should have right access-control
[ https://issues.apache.org/jira/browse/MAPREDUCE-842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12741863#action_12741863 ] Hadoop QA commented on MAPREDUCE-842: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12416158/HADOOP-4491-20090811.txt against trunk revision 803049. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 50 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/466/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/466/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/466/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/466/console This message is automatically generated. Per-job local data on the TaskTracker node should have right access-control --- Key: MAPREDUCE-842 URL: https://issues.apache.org/jira/browse/MAPREDUCE-842 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: tasktracker Reporter: Arun C Murthy Assignee: Vinod K V Attachments: HADOOP-4491-20090623-common.1.txt, HADOOP-4491-20090623-mapred.1.txt, HADOOP-4491-20090703-common.1.txt, HADOOP-4491-20090703-common.txt, HADOOP-4491-20090703.1.txt, HADOOP-4491-20090703.txt, HADOOP-4491-20090707-common.txt, HADOOP-4491-20090707.txt, HADOOP-4491-20090716-mapred.txt, HADOOP-4491-20090803.1.txt, HADOOP-4491-20090803.txt, HADOOP-4491-20090806.txt, HADOOP-4491-20090807.2.txt, HADOOP-4491-20090810.1.txt, HADOOP-4491-20090810.3.txt, HADOOP-4491-20090811.txt -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-848) TestCapacityScheduler is failing
[ https://issues.apache.org/jira/browse/MAPREDUCE-848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12742283#action_12742283 ] Hadoop QA commented on MAPREDUCE-848: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12416284/MAPREDUCE-848-v1.0.patch against trunk revision 803231. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 6 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/470/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/470/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/470/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/470/console This message is automatically generated. TestCapacityScheduler is failing Key: MAPREDUCE-848 URL: https://issues.apache.org/jira/browse/MAPREDUCE-848 Project: Hadoop Map/Reduce Issue Type: Bug Components: contrib/capacity-sched Affects Versions: 0.21.0 Reporter: Devaraj Das Assignee: Amar Kamat Fix For: 0.21.0 Attachments: MAPREDUCE-848-v1.0.patch Looks like the commit of HADOOP-805 broke the CapacityScheduler testcase. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-842) Per-job local data on the TaskTracker node should have right access-control
[ https://issues.apache.org/jira/browse/MAPREDUCE-842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12742366#action_12742366 ] Hadoop QA commented on MAPREDUCE-842: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12416292/HADOOP-4491-20090812.txt against trunk revision 803231. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 50 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/471/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/471/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/471/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/471/console This message is automatically generated. Per-job local data on the TaskTracker node should have right access-control --- Key: MAPREDUCE-842 URL: https://issues.apache.org/jira/browse/MAPREDUCE-842 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: tasktracker Reporter: Arun C Murthy Assignee: Vinod K V Attachments: HADOOP-4491-20090623-common.1.txt, HADOOP-4491-20090623-mapred.1.txt, HADOOP-4491-20090703-common.1.txt, HADOOP-4491-20090703-common.txt, HADOOP-4491-20090703.1.txt, HADOOP-4491-20090703.txt, HADOOP-4491-20090707-common.txt, HADOOP-4491-20090707.txt, HADOOP-4491-20090716-mapred.txt, HADOOP-4491-20090803.1.txt, HADOOP-4491-20090803.txt, HADOOP-4491-20090806.txt, HADOOP-4491-20090807.2.txt, HADOOP-4491-20090810.1.txt, HADOOP-4491-20090810.3.txt, HADOOP-4491-20090811.txt, HADOOP-4491-20090812.txt -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-867) trunk builds fails as ivy is lookin for avro jar from the local resolver
[ https://issues.apache.org/jira/browse/MAPREDUCE-867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12743142#action_12743142 ] Hadoop QA commented on MAPREDUCE-867: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12416534/MAPREDUCE-867.patch against trunk revision 804078. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/473/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/473/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/473/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/473/console This message is automatically generated. trunk builds fails as ivy is lookin for avro jar from the local resolver Key: MAPREDUCE-867 URL: https://issues.apache.org/jira/browse/MAPREDUCE-867 Project: Hadoop Map/Reduce Issue Type: Bug Components: build Reporter: Giridharan Kesavan Assignee: Giridharan Kesavan Attachments: MAPREDUCE-867.patch -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-478) separate jvm param for mapper and reducer
[ https://issues.apache.org/jira/browse/MAPREDUCE-478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12743181#action_12743181 ] Hadoop QA commented on MAPREDUCE-478: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12416542/MAPREDUCE-478_2_20090813.patch against trunk revision 804128. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 26 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/474/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/474/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/474/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/474/console This message is automatically generated. separate jvm param for mapper and reducer - Key: MAPREDUCE-478 URL: https://issues.apache.org/jira/browse/MAPREDUCE-478 Project: Hadoop Map/Reduce Issue Type: Improvement Reporter: Koji Noguchi Assignee: Arun C Murthy Priority: Minor Fix For: 0.21.0 Attachments: HADOOP-5684_0_20090420.patch, MAPREDUCE-478_0_20090804.patch, MAPREDUCE-478_0_20090804_yhadoop20.patch, MAPREDUCE-478_1_20090806.patch, MAPREDUCE-478_1_20090806_yhadoop20.patch, MAPREDUCE-478_2_20090813.patch, MAPREDUCE-478_2_20090813.patch, MAPREDUCE-478_2_20090813_yhadoop.patch Memory footprint of mapper and reducer can differ. It would be nice if we can pass different jvm param (mapred.child.java.opts) for mappers and reducers. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-852) ExampleDriver is incorrectly set as a Main-Class in tools in build.xml
[
https://issues.apache.org/jira/browse/MAPREDUCE-852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12743482#action_12743482
]
Hadoop QA commented on MAPREDUCE-852:
-
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12416346/m852_20090812.patch
against trunk revision 804284.
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified
tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac
compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of
release audit warnings.
-1 core tests. The patch failed core unit tests.
-1 contrib tests. The patch failed contrib unit tests.
Test results:
http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/476/testReport/
Findbugs warnings:
http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/476/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results:
http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/476/artifact/trunk/build/test/checkstyle-errors.html
Console output:
http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/476/console
This message is automatically generated.
ExampleDriver is incorrectly set as a Main-Class in tools in build.xml
--
Key: MAPREDUCE-852
URL: https://issues.apache.org/jira/browse/MAPREDUCE-852
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: build
Reporter: Tsz Wo (Nicholas), SZE
Assignee: Tsz Wo (Nicholas), SZE
Attachments: m852_20090812.patch
In build.xml,
{code}
target name=examples depends=jar, compile-examples description=Make
the Hadoop examples jar.
...
target name=tools-jar depends=jar, compile-tools
description=Make the Hadoop tools jar.
jar jarfile=${build.dir}/${tools.final.name}.jar
basedir=${build.tools}
manifest
attribute name=Main-Class
value=org/apache/hadoop/examples/ExampleDriver/
/manifest
/jar
/target
{code}
- ExampleDriver should not be a Main-Class of tools
- Should we rename the target name from tools-jar to tools, so that the
name would be consistent with the examples target?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-876) Sqoop import of large tables can time out
[ https://issues.apache.org/jira/browse/MAPREDUCE-876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12743561#action_12743561 ] Hadoop QA commented on MAPREDUCE-876: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12416630/MAPREDUCE-876.2.patch against trunk revision 804284. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 5 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/478/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/478/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/478/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/478/console This message is automatically generated. Sqoop import of large tables can time out - Key: MAPREDUCE-876 URL: https://issues.apache.org/jira/browse/MAPREDUCE-876 Project: Hadoop Map/Reduce Issue Type: Improvement Components: contrib/sqoop Reporter: Aaron Kimball Assignee: Aaron Kimball Attachments: MAPREDUCE-876.2.patch, MAPREDUCE-876.patch Related to MAPREDUCE-875, Sqoop should use a background thread to ensure that progress is being reported while a database does external work for the MapReduce task. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-877) Required avro class are missing in contrib projects
[ https://issues.apache.org/jira/browse/MAPREDUCE-877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12743680#action_12743680 ] Hadoop QA commented on MAPREDUCE-877: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12416637/m877_20090814.patch against trunk revision 804284. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/479/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/479/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/479/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/479/console This message is automatically generated. Required avro class are missing in contrib projects --- Key: MAPREDUCE-877 URL: https://issues.apache.org/jira/browse/MAPREDUCE-877 Project: Hadoop Map/Reduce Issue Type: Bug Components: build Affects Versions: 0.21.0 Reporter: Tsz Wo (Nicholas), SZE Assignee: Tsz Wo (Nicholas), SZE Priority: Blocker Fix For: 0.21.0 Attachments: m877_20090814.patch Ivy setting in mapreduce root is updated but the contrib ivy settings are not. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-878) Rename fair scheduler design doc to fair-scheduler-design-doc.tex and add Apache license header
[ https://issues.apache.org/jira/browse/MAPREDUCE-878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12743753#action_12743753 ] Hadoop QA commented on MAPREDUCE-878: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12416665/mapreduce-878.patch against trunk revision 804284. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/482/console This message is automatically generated. Rename fair scheduler design doc to fair-scheduler-design-doc.tex and add Apache license header --- Key: MAPREDUCE-878 URL: https://issues.apache.org/jira/browse/MAPREDUCE-878 Project: Hadoop Map/Reduce Issue Type: Task Components: contrib/fair-share Reporter: Matei Zaharia Assignee: Matei Zaharia Priority: Trivial Attachments: mapreduce-878.patch As suggested by Tsz Wo Tze in MAPREDUCE-706. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-877) Required avro class are missing in contrib projects
[ https://issues.apache.org/jira/browse/MAPREDUCE-877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12743783#action_12743783 ] Hadoop QA commented on MAPREDUCE-877: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12416663/m877_20090815.patch against trunk revision 804284. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/483/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/483/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/483/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/483/console This message is automatically generated. Required avro class are missing in contrib projects --- Key: MAPREDUCE-877 URL: https://issues.apache.org/jira/browse/MAPREDUCE-877 Project: Hadoop Map/Reduce Issue Type: Bug Components: build Affects Versions: 0.21.0 Reporter: Tsz Wo (Nicholas), SZE Assignee: Tsz Wo (Nicholas), SZE Priority: Blocker Fix For: 0.21.0 Attachments: m877_20090814.patch, m877_20090815.patch Ivy setting in mapreduce root is updated but the contrib ivy settings are not. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-870) Clean up the job Retire code
[ https://issues.apache.org/jira/browse/MAPREDUCE-870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12744017#action_12744017 ] Hadoop QA commented on MAPREDUCE-870: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12416751/870_v2.patch against trunk revision 804865. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 7 new or modified tests. -1 javadoc. The javadoc tool appears to have generated 1 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/484/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/484/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/484/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/484/console This message is automatically generated. Clean up the job Retire code Key: MAPREDUCE-870 URL: https://issues.apache.org/jira/browse/MAPREDUCE-870 Project: Hadoop Map/Reduce Issue Type: Sub-task Reporter: Sharad Agarwal Assignee: Sharad Agarwal Attachments: 870_v1.patch, 870_v2.patch Currently completed job's full data structures are kept in memory based on mapred.jobtracker.completeuserjobs.maximum, mapred.jobtracker.retirejob.interval.min, mapred.jobtracker.retirejob.interval and mapred.jobtracker.retirejob.check settings. These controls are not much useful now since MAPREDUCE-817 introduced a cache for keeping just the very basic info of the completed job. These settings should be removed and the job should be purged as soon as the history files are available in HDFS. Going forward, clients can read the history files if they need to drill down into more information (MAPREDUCE-864). -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-777) A method for finding and tracking jobs from the new API
[ https://issues.apache.org/jira/browse/MAPREDUCE-777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12744115#action_12744115 ] Hadoop QA commented on MAPREDUCE-777: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12416762/patch-777-1.txt against trunk revision 804865. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 9 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. -1 javac. The applied patch generated 2272 javac compiler warnings (more than the trunk's current 2232 warnings). +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/485/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/485/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/485/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/485/console This message is automatically generated. A method for finding and tracking jobs from the new API --- Key: MAPREDUCE-777 URL: https://issues.apache.org/jira/browse/MAPREDUCE-777 Project: Hadoop Map/Reduce Issue Type: New Feature Components: client Reporter: Owen O'Malley Assignee: Amareshwari Sriramadasu Fix For: 0.21.0 Attachments: patch-777-1.txt, patch-777.txt We need to create a replacement interface for the JobClient API in the new interface. In particular, the user needs to be able to query and track jobs that were launched by other processes. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-773) LineRecordReader can report non-zero progress while it is processing a compressed stream
[ https://issues.apache.org/jira/browse/MAPREDUCE-773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12744224#action_12744224 ] Hadoop QA commented on MAPREDUCE-773: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12416766/773.3.patch against trunk revision 805081. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/486/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/486/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/486/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/486/console This message is automatically generated. LineRecordReader can report non-zero progress while it is processing a compressed stream Key: MAPREDUCE-773 URL: https://issues.apache.org/jira/browse/MAPREDUCE-773 Project: Hadoop Map/Reduce Issue Type: Bug Components: task Reporter: Devaraj Das Assignee: Devaraj Das Fix For: 0.21.0 Attachments: 773.2.patch, 773.3.patch, 773.patch, 773.patch Currently, the LineRecordReader returns 0.0 from getProgress() for most inputs (since the end of the filesplit is set to Long.MAX_VALUE for compressed inputs). This can be improved to return a non-zero progress even for compressed streams (though it may not be very reflective of the actual progress). -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-874) The name PiEstimator is misleading
[ https://issues.apache.org/jira/browse/MAPREDUCE-874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12744371#action_12744371 ] Hadoop QA commented on MAPREDUCE-874: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12416787/m874_20090817_hudson.patch against trunk revision 805081. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/487/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/487/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/487/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/487/console This message is automatically generated. The name PiEstimator is misleading Key: MAPREDUCE-874 URL: https://issues.apache.org/jira/browse/MAPREDUCE-874 Project: Hadoop Map/Reduce Issue Type: Improvement Components: examples Reporter: Tsz Wo (Nicholas), SZE Assignee: Tsz Wo (Nicholas), SZE Priority: Minor Attachments: m874_20090814.patch, m874_20090817.patch, m874_20090817_hudson.patch The PiEstimator example mainly illustrates evaluating arbitrary integrals numerically by Map/Reduce jobs but not focusing on computing Pi. The name PiEstimator is misleading. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-430) Task stuck in cleanup with OutOfMemoryErrors
[ https://issues.apache.org/jira/browse/MAPREDUCE-430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=1271#action_1271 ] Hadoop QA commented on MAPREDUCE-430: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12416767/MAPREDUCE-430-v1.7.patch against trunk revision 805081. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/488/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/488/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/488/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/488/console This message is automatically generated. Task stuck in cleanup with OutOfMemoryErrors Key: MAPREDUCE-430 URL: https://issues.apache.org/jira/browse/MAPREDUCE-430 Project: Hadoop Map/Reduce Issue Type: Bug Reporter: Amareshwari Sriramadasu Assignee: Amar Kamat Fix For: 0.20.1 Attachments: MAPREDUCE-430-v1.6-branch-0.20.patch, MAPREDUCE-430-v1.6.patch, MAPREDUCE-430-v1.7.patch Obesrved a task with OutOfMemory error, stuck in cleanup. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-476) extend DistributedCache to work locally (LocalJobRunner)
[ https://issues.apache.org/jira/browse/MAPREDUCE-476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12744516#action_12744516 ] Hadoop QA commented on MAPREDUCE-476: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12416836/MAPREDUCE-476-20090818.txt against trunk revision 805324. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 14 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 2 new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/489/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/489/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/489/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/489/console This message is automatically generated. extend DistributedCache to work locally (LocalJobRunner) Key: MAPREDUCE-476 URL: https://issues.apache.org/jira/browse/MAPREDUCE-476 Project: Hadoop Map/Reduce Issue Type: Improvement Reporter: sam rash Assignee: Philip Zeyliger Priority: Minor Attachments: HADOOP-2914-v1-full.patch, HADOOP-2914-v1-since-4041.patch, HADOOP-2914-v2.patch, HADOOP-2914-v3.patch, MAPREDUCE-476-20090814.1.txt, MAPREDUCE-476-20090818.txt, MAPREDUCE-476-v2-vs-v3.patch, MAPREDUCE-476-v2-vs-v3.try2.patch, MAPREDUCE-476-v2-vs-v4.txt, MAPREDUCE-476-v2.patch, MAPREDUCE-476-v3.patch, MAPREDUCE-476-v3.try2.patch, MAPREDUCE-476-v4-requires-MR711.patch, MAPREDUCE-476-v5-requires-MR711.patch, MAPREDUCE-476.patch The DistributedCache does not work locally when using the outlined recipe at http://hadoop.apache.org/core/docs/r0.16.0/api/org/apache/hadoop/filecache/DistributedCache.html Ideally, LocalJobRunner would take care of populating the JobConf and copying remote files to the local file sytem (http, assume hdfs = default fs = local fs when doing local development. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-885) More efficient SQL queries for DBInputFormat
[ https://issues.apache.org/jira/browse/MAPREDUCE-885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12744809#action_12744809 ] Hadoop QA commented on MAPREDUCE-885: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12416936/MAPREDUCE-885.patch against trunk revision 805324. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 4 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/490/console This message is automatically generated. More efficient SQL queries for DBInputFormat Key: MAPREDUCE-885 URL: https://issues.apache.org/jira/browse/MAPREDUCE-885 Project: Hadoop Map/Reduce Issue Type: Improvement Reporter: Aaron Kimball Assignee: Aaron Kimball Attachments: MAPREDUCE-885.patch DBInputFormat generates InputSplits by counting the available rows in a table, and selecting subsections of the table via the LIMIT and OFFSET SQL keywords. These are only meaningful in an ordered context, so the query also includes an ORDER BY clause on an index column. The resulting queries are often inefficient and require full table scans. Actually using multiple mappers with these queries can lead to O(n^2) behavior in the database, where n is the number of splits. Attempting to use parallelism with these queries is counter-productive. A better mechanism is to organize splits based on data values themselves, which can be performed in the WHERE clause, allowing for index range scans of tables, and can better exploit parallelism in the database. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-832) Too many WARN messages about deprecated memorty config variables in JobTacker log
[ https://issues.apache.org/jira/browse/MAPREDUCE-832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12745014#action_12745014 ] Hadoop QA commented on MAPREDUCE-832: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12416966/mapreduce-832.patch against trunk revision 805324. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/493/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/493/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/493/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/493/console This message is automatically generated. Too many WARN messages about deprecated memorty config variables in JobTacker log - Key: MAPREDUCE-832 URL: https://issues.apache.org/jira/browse/MAPREDUCE-832 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 0.20.1 Reporter: Karam Singh Assignee: rahul k singh Attachments: mapreduce-832-20.patch, mapreduce-832.patch When user submit a mapred job using old memory config vairiable (mapred.task.maxmem) followinig message too many times in JobTracker logs -: [ WARN org.apache.hadoop.mapred.JobConf: The variable mapred.task.maxvmem is no longer used instead use mapred.job.map.memory.mb and mapred.job.reduce.memory.mb ] -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-777) A method for finding and tracking jobs from the new API
[ https://issues.apache.org/jira/browse/MAPREDUCE-777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12745124#action_12745124 ] Hadoop QA commented on MAPREDUCE-777: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12416974/patch-777-2.txt against trunk revision 805324. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 9 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. -1 javac. The applied patch generated 2272 javac compiler warnings (more than the trunk's current 2232 warnings). +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/494/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/494/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/494/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/494/console This message is automatically generated. A method for finding and tracking jobs from the new API --- Key: MAPREDUCE-777 URL: https://issues.apache.org/jira/browse/MAPREDUCE-777 Project: Hadoop Map/Reduce Issue Type: New Feature Components: client Reporter: Owen O'Malley Assignee: Amareshwari Sriramadasu Fix For: 0.21.0 Attachments: patch-777-1.txt, patch-777-2.txt, patch-777.txt We need to create a replacement interface for the JobClient API in the new interface. In particular, the user needs to be able to query and track jobs that were launched by other processes. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-834) When TaskTracker config use old memory management values its memory monitoring is diabled.
[ https://issues.apache.org/jira/browse/MAPREDUCE-834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12745375#action_12745375 ] Hadoop QA commented on MAPREDUCE-834: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12417108/mapreduce-834-4.patch against trunk revision 806065. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/496/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/496/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/496/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/496/console This message is automatically generated. When TaskTracker config use old memory management values its memory monitoring is diabled. -- Key: MAPREDUCE-834 URL: https://issues.apache.org/jira/browse/MAPREDUCE-834 Project: Hadoop Map/Reduce Issue Type: Bug Reporter: Karam Singh Assignee: Sreekanth Ramakrishnan Attachments: mapreduce-834-1.patch, mapreduce-834-2.patch, mapreduce-834-3.patch, mapreduce-834-4.patch TaskTracker memory config values -: mapred.tasktracker.vmem.reserved=8589934592 mapred.task.default.maxvmem=2147483648 mapred.task.limit.maxvmem=4294967296 mapred.tasktracker.pmem.reserved=2147483648 TaskTracker start as -: 2009-08-05 12:39:03,308 WARN org.apache.hadoop.mapred.TaskTracker: The variable mapred.tasktracker.vmem.reserved is no longer used 2009-08-05 12:39:03,308 WARN org.apache.hadoop.mapred.TaskTracker: The variable mapred.tasktracker.pmem.reserved is no longer used 2009-08-05 12:39:03,308 WARN org.apache.hadoop.mapred.TaskTracker: The variable mapred.task.default.maxvmem is no longer used 2009-08-05 12:39:03,308 WARN org.apache.hadoop.mapred.TaskTracker: The variable mapred.task.limit.maxvmem is no longer used 2009-08-05 12:39:03,308 INFO org.apache.hadoop.mapred.TaskTracker: Starting thread: Map-events fetcher for all reduce tasks on tracker_name 2009-08-05 12:39:03,309 INFO org.apache.hadoop.mapred.TaskTracker: Using MemoryCalculatorPlugin : org.apache.hadoop.util.linuxmemorycalculatorplu...@19be4777 2009-08-05 12:39:03,311 WARN org.apache.hadoop.mapred.TaskTracker: TaskTracker's totalMemoryAllottedForTasks is -1. TaskMemoryManager is disabled. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-370) Change org.apache.hadoop.mapred.lib.MultipleOutputs to use new api.
[ https://issues.apache.org/jira/browse/MAPREDUCE-370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12745440#action_12745440 ] Hadoop QA commented on MAPREDUCE-370: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12417117/patch-370-1.txt against trunk revision 806065. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/497/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/497/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/497/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/497/console This message is automatically generated. Change org.apache.hadoop.mapred.lib.MultipleOutputs to use new api. --- Key: MAPREDUCE-370 URL: https://issues.apache.org/jira/browse/MAPREDUCE-370 Project: Hadoop Map/Reduce Issue Type: Sub-task Reporter: Amareshwari Sriramadasu Assignee: Amareshwari Sriramadasu Fix For: 0.21.0 Attachments: patch-370-1.txt, patch-370.txt -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-871) Job/Task local files have incorrect group ownership set by LinuxTaskController binary
[ https://issues.apache.org/jira/browse/MAPREDUCE-871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12745637#action_12745637 ] Hadoop QA commented on MAPREDUCE-871: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12417121/MAPREDUCE-871-20090820.1.txt against trunk revision 806152. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 6 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/498/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/498/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/498/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/498/console This message is automatically generated. Job/Task local files have incorrect group ownership set by LinuxTaskController binary - Key: MAPREDUCE-871 URL: https://issues.apache.org/jira/browse/MAPREDUCE-871 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: tasktracker Reporter: Vinod K V Assignee: Vinod K V Attachments: MAPREDUCE-871-20090820.1.txt HADOOP-4491 fixed the secure permissions of local files on a TT. While testing HADOOP-4491 on a cluster, [~karams] found out a bug. All the files/dirs have should be owned by the group corresponding to the group owner of the task-controller binary (via using getegid()) which in turn is a special group to which *only* TT user belongs. HADOOP-4491 incorrectly set it to primary group of the TT via getgid(), and not the special group. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-870) Clean up the job Retire code
[ https://issues.apache.org/jira/browse/MAPREDUCE-870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12745736#action_12745736 ] Hadoop QA commented on MAPREDUCE-870: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12417097/870_v3.patch against trunk revision 806288. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 13 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/499/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/499/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/499/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/499/console This message is automatically generated. Clean up the job Retire code Key: MAPREDUCE-870 URL: https://issues.apache.org/jira/browse/MAPREDUCE-870 Project: Hadoop Map/Reduce Issue Type: Sub-task Reporter: Sharad Agarwal Assignee: Sharad Agarwal Attachments: 870_v1.patch, 870_v2.patch, 870_v3.patch Currently completed job's full data structures are kept in memory based on mapred.jobtracker.completeuserjobs.maximum, mapred.jobtracker.retirejob.interval.min, mapred.jobtracker.retirejob.interval and mapred.jobtracker.retirejob.check settings. These controls are not much useful now since MAPREDUCE-817 introduced a cache for keeping just the very basic info of the completed job. These settings should be removed and the job should be purged as soon as the history files are available in HDFS. Going forward, clients can read the history files if they need to drill down into more information (MAPREDUCE-864). -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-905) Add Eclipse launch tasks for MapReduce
[ https://issues.apache.org/jira/browse/MAPREDUCE-905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12746475#action_12746475 ] Hadoop QA commented on MAPREDUCE-905: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12417341/MAPREDUCE-905.patch against trunk revision 806764. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 4 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/506/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/506/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/506/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/506/console This message is automatically generated. Add Eclipse launch tasks for MapReduce -- Key: MAPREDUCE-905 URL: https://issues.apache.org/jira/browse/MAPREDUCE-905 Project: Hadoop Map/Reduce Issue Type: Improvement Environment: Eclipse 3.5 Reporter: Philip Zeyliger Priority: Minor Attachments: MAPREDUCE-905.patch This is a revival of HADOOP-5911, but only for the MR project. Eclipse has a notion of run configuration, which encapsulates what's needed to run or debug an application. I use this quite a bit to start various Hadoop daemons in debug mode, with breakpoints set, to inspect state and what not. This is simply configuration, so no tests are provided. After running ant eclipse-files and refreshing your project, you should see entries in the Run pulldown. There's a template for testing a specific test, and also templates to run all the tests, the job tracker, and a task tracker. It's likely that some parameters need to be further tweaked to have the same behavior as ant test, but for most tests, this works. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-370) Change org.apache.hadoop.mapred.lib.MultipleOutputs to use new api.
[ https://issues.apache.org/jira/browse/MAPREDUCE-370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12746887#action_12746887 ] Hadoop QA commented on MAPREDUCE-370: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12417469/patch-370-2.txt against trunk revision 807123. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/508/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/508/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/508/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/508/console This message is automatically generated. Change org.apache.hadoop.mapred.lib.MultipleOutputs to use new api. --- Key: MAPREDUCE-370 URL: https://issues.apache.org/jira/browse/MAPREDUCE-370 Project: Hadoop Map/Reduce Issue Type: Sub-task Reporter: Amareshwari Sriramadasu Assignee: Amareshwari Sriramadasu Fix For: 0.21.0 Attachments: patch-370-1.txt, patch-370-2.txt, patch-370.txt -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-910) MRUnit should support counters
[ https://issues.apache.org/jira/browse/MAPREDUCE-910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12747162#action_12747162 ] Hadoop QA commented on MAPREDUCE-910: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12417505/MAPREDUCE-910.patch against trunk revision 807165. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 13 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/510/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/510/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/510/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/510/console This message is automatically generated. MRUnit should support counters -- Key: MAPREDUCE-910 URL: https://issues.apache.org/jira/browse/MAPREDUCE-910 Project: Hadoop Map/Reduce Issue Type: Improvement Reporter: Aaron Kimball Assignee: Aaron Kimball Attachments: MAPREDUCE-910.patch incrCounter() is currently a dummy stub method in MRUnit that does nothing. Would be good for the mock reporter/context implementations to support counters. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-775) Add input/output formatters for Vertica clustered ADBMS.
[ https://issues.apache.org/jira/browse/MAPREDUCE-775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12747222#action_12747222 ] Hadoop QA commented on MAPREDUCE-775: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12415514/MAPREDUCE-775.patch against trunk revision 807165. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 16 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/511/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/511/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/511/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/511/console This message is automatically generated. Add input/output formatters for Vertica clustered ADBMS. Key: MAPREDUCE-775 URL: https://issues.apache.org/jira/browse/MAPREDUCE-775 Project: Hadoop Map/Reduce Issue Type: New Feature Reporter: Omer Trajman Fix For: 0.21.0 Attachments: MAPREDUCE-775.patch Add native support for Vertica as an input or output format taking advantage of parallel read and write properties of the DBMS. On the input side allow for parametrized queries (a la prepared statements) and create a split for each combination of parameters. Also support the parameter list to be generated from a sql statement. For example - return metrics for all dimensions that meet criteria X with one input split for each dimension. Divide the read among any number of hosts in the Vertica cluster. On the output side, support Vertica streaming load to any number of hosts in the Vertica cluster. Output may be to a different cluster than input. Also includes Input and Output formatters that support streaming interface. Code has been tested and run on live systems under 19 and 20. Patch for 21 with new API will be ready end of this week. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-875) Make DBRecordReader execute queries lazily
[ https://issues.apache.org/jira/browse/MAPREDUCE-875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12747314#action_12747314 ] Hadoop QA commented on MAPREDUCE-875: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12416939/MAPREDUCE-875.2.patch against trunk revision 807165. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/513/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/513/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/513/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/513/console This message is automatically generated. Make DBRecordReader execute queries lazily -- Key: MAPREDUCE-875 URL: https://issues.apache.org/jira/browse/MAPREDUCE-875 Project: Hadoop Map/Reduce Issue Type: Improvement Reporter: Aaron Kimball Assignee: Aaron Kimball Attachments: MAPREDUCE-875.2.patch, MAPREDUCE-875.patch DBInputFormat's DBRecordReader executes the user's SQL query in the constructor. If the query is long-running, this can cause task timeout. The user is unable to spawn a background thread (e.g., in a MapRunnable) to inform Hadoop of on-going progress. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-911) TestTaskFail fail sometimes
[ https://issues.apache.org/jira/browse/MAPREDUCE-911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12747454#action_12747454 ] Hadoop QA commented on MAPREDUCE-911: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12417581/patch-911.txt against trunk revision 807543. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. -1 release audit. The applied patch generated 202 release audit warnings (more than the trunk's current 0 warnings). -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/515/testReport/ Release audit warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/515/artifact/trunk/patchprocess/releaseAuditDiffWarnings.txt Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/515/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/515/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/515/console This message is automatically generated. TestTaskFail fail sometimes --- Key: MAPREDUCE-911 URL: https://issues.apache.org/jira/browse/MAPREDUCE-911 Project: Hadoop Map/Reduce Issue Type: Bug Components: test Reporter: Amareshwari Sriramadasu Assignee: Amareshwari Sriramadasu Fix For: 0.20.1 Attachments: faillog.txt, patch-911-0.20.txt, patch-911.txt TestTaskFail fail sometimes with following error : junit.framework.AssertionFailedError at org.apache.hadoop.mapred.TestTaskFail.validateJob(TestTaskFail.java:136) at org.apache.hadoop.mapred.TestTaskFail.testWithDFS(TestTaskFail.java:181) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-824) Support a hierarchy of queues in the capacity scheduler
[ https://issues.apache.org/jira/browse/MAPREDUCE-824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12747800#action_12747800 ] Hadoop QA commented on MAPREDUCE-824: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12417611/MAPREDUCE-824-6.patch against trunk revision 807640. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 16 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/518/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/518/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/518/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/518/console This message is automatically generated. Support a hierarchy of queues in the capacity scheduler --- Key: MAPREDUCE-824 URL: https://issues.apache.org/jira/browse/MAPREDUCE-824 Project: Hadoop Map/Reduce Issue Type: New Feature Components: contrib/capacity-sched Reporter: Hemanth Yamijala Attachments: HADOOP-824-1.patch, HADOOP-824-2.patch, HADOOP-824-3.patch, HADOOP-824-4.patch, HADOOP-824-5.patch, MAPREDUCE-824-6.patch, MAPREDUCE-824-7.patch Currently in Capacity Scheduler, cluster capacity is divided among the queues based on the queue capacity. These queues typically represent an organization and the capacity of the queue represents the capacity the organization is entitled to. Most organizations are large and need to divide their capacity among sub-organizations they have. Or they may want to divide the capacity based on a category or type of jobs they run. This JIRA covers the requirements and other details to provide the above feature. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-906) Updated Sqoop documentation
[ https://issues.apache.org/jira/browse/MAPREDUCE-906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12747885#action_12747885 ] Hadoop QA commented on MAPREDUCE-906: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12417639/MAPREDUCE-906.2.patch against trunk revision 807908. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. -1 release audit. The applied patch generated 203 release audit warnings (more than the trunk's current 202 warnings). -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/519/testReport/ Release audit warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/519/artifact/trunk/patchprocess/releaseAuditDiffWarnings.txt Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/519/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/519/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/519/console This message is automatically generated. Updated Sqoop documentation --- Key: MAPREDUCE-906 URL: https://issues.apache.org/jira/browse/MAPREDUCE-906 Project: Hadoop Map/Reduce Issue Type: Improvement Components: contrib/sqoop Reporter: Aaron Kimball Assignee: Aaron Kimball Attachments: MAPREDUCE-906.2.patch, MAPREDUCE-906.patch Here's the latest documentation for Sqoop, in both user-guide and manpage form. Built with asciidoc. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-824) Support a hierarchy of queues in the capacity scheduler
[ https://issues.apache.org/jira/browse/MAPREDUCE-824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12747987#action_12747987 ] Hadoop QA commented on MAPREDUCE-824: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12417701/MAPREDUCE-824-7.patch against trunk revision 807954. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 16 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/520/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/520/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/520/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/520/console This message is automatically generated. Support a hierarchy of queues in the capacity scheduler --- Key: MAPREDUCE-824 URL: https://issues.apache.org/jira/browse/MAPREDUCE-824 Project: Hadoop Map/Reduce Issue Type: New Feature Components: contrib/capacity-sched Reporter: Hemanth Yamijala Attachments: HADOOP-824-1.patch, HADOOP-824-2.patch, HADOOP-824-3.patch, HADOOP-824-4.patch, HADOOP-824-5.patch, MAPREDUCE-824-6.patch, MAPREDUCE-824-7.patch Currently in Capacity Scheduler, cluster capacity is divided among the queues based on the queue capacity. These queues typically represent an organization and the capacity of the queue represents the capacity the organization is entitled to. Most organizations are large and need to divide their capacity among sub-organizations they have. Or they may want to divide the capacity based on a category or type of jobs they run. This JIRA covers the requirements and other details to provide the above feature. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-644) distcp does not skip copying file if we are updating single file
[ https://issues.apache.org/jira/browse/MAPREDUCE-644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12748305#action_12748305 ] Hadoop QA commented on MAPREDUCE-644: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12411201/d_singlefile_update.patch against trunk revision 808082. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/524/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/524/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/524/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/524/console This message is automatically generated. distcp does not skip copying file if we are updating single file Key: MAPREDUCE-644 URL: https://issues.apache.org/jira/browse/MAPREDUCE-644 Project: Hadoop Map/Reduce Issue Type: Bug Components: distcp Reporter: Ravi Gummadi Assignee: Ravi Gummadi Attachments: d_singlefile_update.patch distcp doesn't skip copying file when we do -update on single file if the destfile already exists. When we do hadoop distcp -update srcfilename destfilename it seems to be comparing checksums of srcfilename and destfilename/srcfilename and so skip is not done. It should compare checksums of srcfilename and destfilename. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-645) When disctp is used to overwrite a file, it should return immediately with an error message
[ https://issues.apache.org/jira/browse/MAPREDUCE-645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12748443#action_12748443 ] Hadoop QA commented on MAPREDUCE-645: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12412151/d_645.patch against trunk revision 808351. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/528/console This message is automatically generated. When disctp is used to overwrite a file, it should return immediately with an error message --- Key: MAPREDUCE-645 URL: https://issues.apache.org/jira/browse/MAPREDUCE-645 Project: Hadoop Map/Reduce Issue Type: Bug Components: distcp Reporter: Ramya R Assignee: Ravi Gummadi Priority: Minor Attachments: d_645.patch, distcp.txt When disctp is triggered to copy a directory to an already existing file, it just shows a copy failed error message after 4 attempts without showing any useful error message. This is extremely time consuming on a large cluster and especially when the directory being copied contains several sub-directories. Instead, it would be an improvement if distcp could return immediately displaying a useful error message when an user attempts such an operation. (This is an unlikely situation but still a valid test case) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-751) Rumen: a tool to extract job characterization data from job tracker logs
[ https://issues.apache.org/jira/browse/MAPREDUCE-751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12748579#action_12748579 ] Hadoop QA commented on MAPREDUCE-751: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12417832/mapreduce-751-20090826.patch against trunk revision 808351. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 37 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. -1 javac. The applied patch generated 2226 javac compiler warnings (more than the trunk's current 2220 warnings). +1 findbugs. The patch does not introduce any new Findbugs warnings. -1 release audit. The applied patch generated 215 release audit warnings (more than the trunk's current 202 warnings). -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/530/testReport/ Release audit warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/530/artifact/trunk/patchprocess/releaseAuditDiffWarnings.txt Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/530/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/530/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/530/console This message is automatically generated. Rumen: a tool to extract job characterization data from job tracker logs Key: MAPREDUCE-751 URL: https://issues.apache.org/jira/browse/MAPREDUCE-751 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 0.20.1, 0.21.0 Reporter: Dick King Assignee: Dick King Fix For: 0.21.0 Attachments: 2009-08-19--1030.patch, 2009-08-26--1513-patch.patch, mapreduce-751--2009-07-23.patch, mapreduce-751-20090826.patch We propose a new map/reduce component, rumen, which can be used to process job history logs to produce any or all of the following: * Retrospective info describing the statistical behavior of the amount of time it would have taken to launch a job into a certain percentage of the number of mapper slots in the log's cluster, given the load over the period covered by the log * Statistical info as to the runtimes and shuffle times, etc. of the tasks and jobs covered by the log * files describing detailed job trace information, and the network topology as inferred from the host locations and rack IDs that arise in the job tracker log. In addition to this facility, rumen includes readers for this information to return job and detailed task information to other tools. These other tools include a more advanced version of gridmix, and also includes mumak: see blocked issues. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-918) Test hsqldb server should be memory-only.
[
https://issues.apache.org/jira/browse/MAPREDUCE-918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12748653#action_12748653
]
Hadoop QA commented on MAPREDUCE-918:
-
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12417852/MAPREDUCE-918.patch
against trunk revision 808650.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac
compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of
release audit warnings.
-1 core tests. The patch failed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results:
http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/531/testReport/
Findbugs warnings:
http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/531/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results:
http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/531/artifact/trunk/build/test/checkstyle-errors.html
Console output:
http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/531/console
This message is automatically generated.
Test hsqldb server should be memory-only.
-
Key: MAPREDUCE-918
URL: https://issues.apache.org/jira/browse/MAPREDUCE-918
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: contrib/sqoop
Reporter: Aaron Kimball
Assignee: Aaron Kimball
Attachments: MAPREDUCE-918.patch
Sqoop launches a standalone hsqldb server for unit tests, but it currently
writes its database to disk and uses a connect string of {{//localhost}}. If
multiple test instances are running concurrently, one test server may serve
to the other instance of the unit tests, causing race conditions.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-885) More efficient SQL queries for DBInputFormat
[ https://issues.apache.org/jira/browse/MAPREDUCE-885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12748655#action_12748655 ] Hadoop QA commented on MAPREDUCE-885: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12417849/MAPREDUCE-885.3.patch against trunk revision 808730. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/532/console This message is automatically generated. More efficient SQL queries for DBInputFormat Key: MAPREDUCE-885 URL: https://issues.apache.org/jira/browse/MAPREDUCE-885 Project: Hadoop Map/Reduce Issue Type: Improvement Reporter: Aaron Kimball Assignee: Aaron Kimball Attachments: MAPREDUCE-885.2.patch, MAPREDUCE-885.3.patch, MAPREDUCE-885.patch DBInputFormat generates InputSplits by counting the available rows in a table, and selecting subsections of the table via the LIMIT and OFFSET SQL keywords. These are only meaningful in an ordered context, so the query also includes an ORDER BY clause on an index column. The resulting queries are often inefficient and require full table scans. Actually using multiple mappers with these queries can lead to O(n^2) behavior in the database, where n is the number of splits. Attempting to use parallelism with these queries is counter-productive. A better mechanism is to organize splits based on data values themselves, which can be performed in the WHERE clause, allowing for index range scans of tables, and can better exploit parallelism in the database. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-370) Change org.apache.hadoop.mapred.lib.MultipleOutputs to use new api.
[ https://issues.apache.org/jira/browse/MAPREDUCE-370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12748701#action_12748701 ] Hadoop QA commented on MAPREDUCE-370: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12417954/patch-370-3.txt against trunk revision 808730. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/533/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/533/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/533/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/533/console This message is automatically generated. Change org.apache.hadoop.mapred.lib.MultipleOutputs to use new api. --- Key: MAPREDUCE-370 URL: https://issues.apache.org/jira/browse/MAPREDUCE-370 Project: Hadoop Map/Reduce Issue Type: Sub-task Reporter: Amareshwari Sriramadasu Assignee: Amareshwari Sriramadasu Fix For: 0.21.0 Attachments: patch-370-1.txt, patch-370-2.txt, patch-370-3.txt, patch-370.txt -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-898) Change DistributedCache to use new api.
[ https://issues.apache.org/jira/browse/MAPREDUCE-898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12748800#action_12748800 ] Hadoop QA commented on MAPREDUCE-898: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12417958/patch-898.txt against trunk revision 808730. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. -1 javac. The applied patch generated 2230 javac compiler warnings (more than the trunk's current 2226 warnings). +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/534/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/534/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/534/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/534/console This message is automatically generated. Change DistributedCache to use new api. --- Key: MAPREDUCE-898 URL: https://issues.apache.org/jira/browse/MAPREDUCE-898 Project: Hadoop Map/Reduce Issue Type: Sub-task Reporter: Amareshwari Sriramadasu Assignee: Amareshwari Sriramadasu Fix For: 0.21.0 Attachments: patch-898.txt -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-885) More efficient SQL queries for DBInputFormat
[ https://issues.apache.org/jira/browse/MAPREDUCE-885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12749059#action_12749059 ] Hadoop QA commented on MAPREDUCE-885: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12418034/MAPREDUCE-885.5.patch against trunk revision 808730. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 4 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 3 new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/535/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/535/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/535/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/535/console This message is automatically generated. More efficient SQL queries for DBInputFormat Key: MAPREDUCE-885 URL: https://issues.apache.org/jira/browse/MAPREDUCE-885 Project: Hadoop Map/Reduce Issue Type: Improvement Reporter: Aaron Kimball Assignee: Aaron Kimball Attachments: MAPREDUCE-885.2.patch, MAPREDUCE-885.3.patch, MAPREDUCE-885.4.patch, MAPREDUCE-885.5.patch, MAPREDUCE-885.patch DBInputFormat generates InputSplits by counting the available rows in a table, and selecting subsections of the table via the LIMIT and OFFSET SQL keywords. These are only meaningful in an ordered context, so the query also includes an ORDER BY clause on an index column. The resulting queries are often inefficient and require full table scans. Actually using multiple mappers with these queries can lead to O(n^2) behavior in the database, where n is the number of splits. Attempting to use parallelism with these queries is counter-productive. A better mechanism is to organize splits based on data values themselves, which can be performed in the WHERE clause, allowing for index range scans of tables, and can better exploit parallelism in the database. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-898) Change DistributedCache to use new api.
[ https://issues.apache.org/jira/browse/MAPREDUCE-898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12749447#action_12749447 ] Hadoop QA commented on MAPREDUCE-898: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12417958/patch-898.txt against trunk revision 808730. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. -1 javac. The applied patch generated 2230 javac compiler warnings (more than the trunk's current 2226 warnings). +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/25/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/25/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/25/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/25/console This message is automatically generated. Change DistributedCache to use new api. --- Key: MAPREDUCE-898 URL: https://issues.apache.org/jira/browse/MAPREDUCE-898 Project: Hadoop Map/Reduce Issue Type: Sub-task Reporter: Amareshwari Sriramadasu Assignee: Amareshwari Sriramadasu Fix For: 0.21.0 Attachments: patch-898.txt -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-839) unit test TestMiniMRChildTask fails on mac os-x
[ https://issues.apache.org/jira/browse/MAPREDUCE-839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12749650#action_12749650 ] Hadoop QA commented on MAPREDUCE-839: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12418043/mapreduce-839-20090828.patch against trunk revision 808730. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/27/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/27/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/27/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/27/console This message is automatically generated. unit test TestMiniMRChildTask fails on mac os-x --- Key: MAPREDUCE-839 URL: https://issues.apache.org/jira/browse/MAPREDUCE-839 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 0.21.0 Reporter: Hong Tang Assignee: Hong Tang Priority: Minor Fix For: 0.21.0 Attachments: mapreduce-839-20090828.patch The unit test TestMiniMRChildTask fails on Mac OS-X (10.5.8) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
