[jira] [Created] (MAPREDUCE-4922) Request with multiple data local nodes can cause NPE in AppSchedulingInfo
Sandy Ryza created MAPREDUCE-4922: - Summary: Request with multiple data local nodes can cause NPE in AppSchedulingInfo Key: MAPREDUCE-4922 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4922 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 2.0.2-alpha Reporter: Sandy Ryza Assignee: Sandy Ryza With the way that the schedulers work, each request for a container on a node must consist of 3 ResourceRequests - one on the node, one on the rack, and one with *. AppSchedulingInfo tracks the outstanding requests. When a node is assigned a node-local container, allocateNodeLocal decrements the outstanding requests at each level - node, rack, and *. If the rack requests reach 0, it removes the mapping. A mapreduce task with multiple data local nodes submits multiple container requests, one for each node. It also submits one for each unique rack, and one for *. If there are fewer unique racks than data local nodes, this means that fewer rack-local ResourceRequests will be submitted than node-local ResourceRequests, so the rack-local mapping will be deleted before all the node-local requests are allocated and an NPE will come up the next time a node-local request from that rack is allocated. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-4923) Add toString method to TaggedInputSplit
Sandy Ryza created MAPREDUCE-4923: - Summary: Add toString method to TaggedInputSplit Key: MAPREDUCE-4923 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4923 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv1, mrv2, task Affects Versions: 2.0.2-alpha, 1.1.1 Reporter: Sandy Ryza Assignee: Sandy Ryza Priority: Minor Per MAPREDUCE-3678, map task logs now contain information about the input split being processed. Because TaggedInputSplit has no overridden toString method, nothing useful gets printed out. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-4924) flakey test: org.apache.hadoop.mapred.TestClusterMRNotification.testMR
Robert Kanter created MAPREDUCE-4924:
Summary: flakey test:
org.apache.hadoop.mapred.TestClusterMRNotification.testMR
Key: MAPREDUCE-4924
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4924
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv1
Affects Versions: 2.0.2-alpha
Reporter: Robert Kanter
Priority: Trivial
I occasionally get a failure like this on
{{org.apache.hadoop.mapred.TestClusterMRNotification.testMR}}
{code}
junit.framework.AssertionFailedError: expected:6 but was:4
at junit.framework.Assert.fail(Assert.java:47)
at junit.framework.Assert.failNotEquals(Assert.java:283)
at junit.framework.Assert.assertEquals(Assert.java:64)
at junit.framework.Assert.assertEquals(Assert.java:195)
at junit.framework.Assert.assertEquals(Assert.java:201)
at
org.apache.hadoop.mapred.NotificationTestCase.testMR(NotificationTestCase.java:184)
...
{code}
It looks like a race condition:
{code}
// run a job with FAILED status
System.out.println(UtilsForTests.runJobFail(this.createJobConf(), inDir,
outDir).getID());
Thread.sleep(2000);
assertEquals(6, NotificationServlet.counter);
assertEquals(0, NotificationServlet.failureCounter);
{code}
Instead of sleeping for 2 seconds, we should keep checking the counter and fail
after a timeout. There's a couple of similar places in the test that should be
fixed too.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-4925) Backport MAPREDUCE-4678 to branch-1
Karthik Kambatla created MAPREDUCE-4925: --- Summary: Backport MAPREDUCE-4678 to branch-1 Key: MAPREDUCE-4925 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4925 Project: Hadoop Map/Reduce Issue Type: Bug Components: examples Reporter: Karthik Kambatla Assignee: Karthik Kambatla -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
