[jira] [Updated] (YARN-1572) Low chance to hit NPE issue in AppSchedulingInfo#allocateNodeLocal
[
https://issues.apache.org/jira/browse/YARN-1572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ray Chiang updated YARN-1572:
-
Labels: (was: BB2015-05-TBR)
Low chance to hit NPE issue in AppSchedulingInfo#allocateNodeLocal
--
Key: YARN-1572
URL: https://issues.apache.org/jira/browse/YARN-1572
Project: Hadoop YARN
Issue Type: Bug
Components: scheduler
Affects Versions: 2.3.0
Reporter: Wenwu Peng
Assignee: Wenwu Peng
Attachments: YARN-1572-branch-2.3.0.001.patch, YARN-1572-log.tar.gz,
conf.tar.gz, log.tar.gz
we have lower chance to hit NPE in allocateNodeLocal when run benchmark(hit
4 in 20 times).
{code}
2014-07-31 04:18:19,653 INFO
org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerNode:
Assigned container container_1406794589275_0001_01_21 of capacity
memory:1024, vCores:1 on host datanode10:57281, which has 6 containers,
memory:6144, vCores:6 used and memory:2048, vCores:2 available after
allocation
2014-07-31 04:18:19,654 FATAL
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error in
handling event type NODE_UPDATE to the scheduler
java.lang.NullPointerException
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.AppSchedulingInfo.allocateNodeLocal(AppSchedulingInfo.java:311)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.AppSchedulingInfo.allocate(AppSchedulingInfo.java:268)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.common.fica.FiCaSchedulerApp.allocate(FiCaSchedulerApp.java:136)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.assignContainer(FifoScheduler.java:683)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.assignNodeLocalContainers(FifoScheduler.java:602)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.assignContainersOnNode(FifoScheduler.java:560)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.assignContainers(FifoScheduler.java:488)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.nodeUpdate(FifoScheduler.java:729)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.handle(FifoScheduler.java:774)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.handle(FifoScheduler.java:101)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher$EventProcessor.run(ResourceManager.java:599)
at java.lang.Thread.run(Thread.java:662)
2014-07-31 04:18:19,655 INFO
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Exiting, bbye..
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (YARN-1572) Low chance to hit NPE issue in AppSchedulingInfo#allocateNodeLocal
[
https://issues.apache.org/jira/browse/YARN-1572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allen Wittenauer updated YARN-1572:
---
Labels: BB2015-05-TBR (was: )
Low chance to hit NPE issue in AppSchedulingInfo#allocateNodeLocal
--
Key: YARN-1572
URL: https://issues.apache.org/jira/browse/YARN-1572
Project: Hadoop YARN
Issue Type: Bug
Components: scheduler
Affects Versions: 2.3.0
Reporter: Wenwu Peng
Assignee: Wenwu Peng
Labels: BB2015-05-TBR
Attachments: YARN-1572-branch-2.3.0.001.patch, YARN-1572-log.tar.gz,
conf.tar.gz, log.tar.gz
we have lower chance to hit NPE in allocateNodeLocal when run benchmark(hit
4 in 20 times).
{code}
2014-07-31 04:18:19,653 INFO
org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerNode:
Assigned container container_1406794589275_0001_01_21 of capacity
memory:1024, vCores:1 on host datanode10:57281, which has 6 containers,
memory:6144, vCores:6 used and memory:2048, vCores:2 available after
allocation
2014-07-31 04:18:19,654 FATAL
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error in
handling event type NODE_UPDATE to the scheduler
java.lang.NullPointerException
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.AppSchedulingInfo.allocateNodeLocal(AppSchedulingInfo.java:311)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.AppSchedulingInfo.allocate(AppSchedulingInfo.java:268)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.common.fica.FiCaSchedulerApp.allocate(FiCaSchedulerApp.java:136)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.assignContainer(FifoScheduler.java:683)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.assignNodeLocalContainers(FifoScheduler.java:602)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.assignContainersOnNode(FifoScheduler.java:560)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.assignContainers(FifoScheduler.java:488)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.nodeUpdate(FifoScheduler.java:729)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.handle(FifoScheduler.java:774)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.handle(FifoScheduler.java:101)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher$EventProcessor.run(ResourceManager.java:599)
at java.lang.Thread.run(Thread.java:662)
2014-07-31 04:18:19,655 INFO
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Exiting, bbye..
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (YARN-1572) Low chance to hit NPE issue in AppSchedulingInfo#allocateNodeLocal
[
https://issues.apache.org/jira/browse/YARN-1572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jian He updated YARN-1572:
--
Target Version/s: 2.8.0
Low chance to hit NPE issue in AppSchedulingInfo#allocateNodeLocal
--
Key: YARN-1572
URL: https://issues.apache.org/jira/browse/YARN-1572
Project: Hadoop YARN
Issue Type: Bug
Components: scheduler
Affects Versions: 2.3.0
Reporter: Wenwu Peng
Assignee: Wenwu Peng
Attachments: YARN-1572-branch-2.3.0.001.patch, YARN-1572-log.tar.gz,
conf.tar.gz, log.tar.gz
we have lower chance to hit NPE in allocateNodeLocal when run benchmark(hit
4 in 20 times).
{code}
2014-07-31 04:18:19,653 INFO
org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerNode:
Assigned container container_1406794589275_0001_01_21 of capacity
memory:1024, vCores:1 on host datanode10:57281, which has 6 containers,
memory:6144, vCores:6 used and memory:2048, vCores:2 available after
allocation
2014-07-31 04:18:19,654 FATAL
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error in
handling event type NODE_UPDATE to the scheduler
java.lang.NullPointerException
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.AppSchedulingInfo.allocateNodeLocal(AppSchedulingInfo.java:311)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.AppSchedulingInfo.allocate(AppSchedulingInfo.java:268)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.common.fica.FiCaSchedulerApp.allocate(FiCaSchedulerApp.java:136)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.assignContainer(FifoScheduler.java:683)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.assignNodeLocalContainers(FifoScheduler.java:602)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.assignContainersOnNode(FifoScheduler.java:560)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.assignContainers(FifoScheduler.java:488)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.nodeUpdate(FifoScheduler.java:729)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.handle(FifoScheduler.java:774)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.handle(FifoScheduler.java:101)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher$EventProcessor.run(ResourceManager.java:599)
at java.lang.Thread.run(Thread.java:662)
2014-07-31 04:18:19,655 INFO
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Exiting, bbye..
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (YARN-1572) Low chance to hit NPE issue in AppSchedulingInfo#allocateNodeLocal
[
https://issues.apache.org/jira/browse/YARN-1572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kareem El Gebaly updated YARN-1572:
---
Attachment: YARN-1572-branch-2.3.0.001.patch
Low chance to hit NPE issue in AppSchedulingInfo#allocateNodeLocal
--
Key: YARN-1572
URL: https://issues.apache.org/jira/browse/YARN-1572
Project: Hadoop YARN
Issue Type: Bug
Components: scheduler
Affects Versions: 2.3.0
Reporter: Wenwu Peng
Assignee: Wenwu Peng
Attachments: YARN-1572-branch-2.3.0.001.patch, YARN-1572-log.tar.gz,
conf.tar.gz, log.tar.gz
we have lower chance to hit NPE in allocateNodeLocal when run benchmark(hit
4 in 20 times).
{code}
2014-07-31 04:18:19,653 INFO
org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerNode:
Assigned container container_1406794589275_0001_01_21 of capacity
memory:1024, vCores:1 on host datanode10:57281, which has 6 containers,
memory:6144, vCores:6 used and memory:2048, vCores:2 available after
allocation
2014-07-31 04:18:19,654 FATAL
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error in
handling event type NODE_UPDATE to the scheduler
java.lang.NullPointerException
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.AppSchedulingInfo.allocateNodeLocal(AppSchedulingInfo.java:311)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.AppSchedulingInfo.allocate(AppSchedulingInfo.java:268)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.common.fica.FiCaSchedulerApp.allocate(FiCaSchedulerApp.java:136)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.assignContainer(FifoScheduler.java:683)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.assignNodeLocalContainers(FifoScheduler.java:602)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.assignContainersOnNode(FifoScheduler.java:560)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.assignContainers(FifoScheduler.java:488)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.nodeUpdate(FifoScheduler.java:729)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.handle(FifoScheduler.java:774)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.handle(FifoScheduler.java:101)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher$EventProcessor.run(ResourceManager.java:599)
at java.lang.Thread.run(Thread.java:662)
2014-07-31 04:18:19,655 INFO
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Exiting, bbye..
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (YARN-1572) Low chance to hit NPE issue in AppSchedulingInfo#allocateNodeLocal
[
https://issues.apache.org/jira/browse/YARN-1572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kareem El Gebaly updated YARN-1572:
---
Attachment: (was: 0001-Fix-for-YARN-1572.patch)
Low chance to hit NPE issue in AppSchedulingInfo#allocateNodeLocal
--
Key: YARN-1572
URL: https://issues.apache.org/jira/browse/YARN-1572
Project: Hadoop YARN
Issue Type: Bug
Components: scheduler
Affects Versions: 2.3.0
Reporter: Wenwu Peng
Assignee: Wenwu Peng
Attachments: YARN-1572-log.tar.gz, conf.tar.gz, log.tar.gz
we have lower chance to hit NPE in allocateNodeLocal when run benchmark(hit
4 in 20 times).
{code}
2014-07-31 04:18:19,653 INFO
org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerNode:
Assigned container container_1406794589275_0001_01_21 of capacity
memory:1024, vCores:1 on host datanode10:57281, which has 6 containers,
memory:6144, vCores:6 used and memory:2048, vCores:2 available after
allocation
2014-07-31 04:18:19,654 FATAL
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error in
handling event type NODE_UPDATE to the scheduler
java.lang.NullPointerException
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.AppSchedulingInfo.allocateNodeLocal(AppSchedulingInfo.java:311)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.AppSchedulingInfo.allocate(AppSchedulingInfo.java:268)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.common.fica.FiCaSchedulerApp.allocate(FiCaSchedulerApp.java:136)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.assignContainer(FifoScheduler.java:683)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.assignNodeLocalContainers(FifoScheduler.java:602)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.assignContainersOnNode(FifoScheduler.java:560)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.assignContainers(FifoScheduler.java:488)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.nodeUpdate(FifoScheduler.java:729)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.handle(FifoScheduler.java:774)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.handle(FifoScheduler.java:101)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher$EventProcessor.run(ResourceManager.java:599)
at java.lang.Thread.run(Thread.java:662)
2014-07-31 04:18:19,655 INFO
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Exiting, bbye..
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (YARN-1572) Low chance to hit NPE issue in AppSchedulingInfo#allocateNodeLocal
[
https://issues.apache.org/jira/browse/YARN-1572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kareem El Gebaly updated YARN-1572:
---
Attachment: 0001-Fix-for-YARN-1572.patch
Fix for YARN-1572 applies to 2.3.0
Low chance to hit NPE issue in AppSchedulingInfo#allocateNodeLocal
--
Key: YARN-1572
URL: https://issues.apache.org/jira/browse/YARN-1572
Project: Hadoop YARN
Issue Type: Bug
Components: scheduler
Affects Versions: 2.3.0
Reporter: Wenwu Peng
Assignee: Wenwu Peng
Attachments: 0001-Fix-for-YARN-1572.patch, YARN-1572-log.tar.gz,
conf.tar.gz, log.tar.gz
we have lower chance to hit NPE in allocateNodeLocal when run benchmark(hit
4 in 20 times).
{code}
2014-07-31 04:18:19,653 INFO
org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerNode:
Assigned container container_1406794589275_0001_01_21 of capacity
memory:1024, vCores:1 on host datanode10:57281, which has 6 containers,
memory:6144, vCores:6 used and memory:2048, vCores:2 available after
allocation
2014-07-31 04:18:19,654 FATAL
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error in
handling event type NODE_UPDATE to the scheduler
java.lang.NullPointerException
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.AppSchedulingInfo.allocateNodeLocal(AppSchedulingInfo.java:311)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.AppSchedulingInfo.allocate(AppSchedulingInfo.java:268)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.common.fica.FiCaSchedulerApp.allocate(FiCaSchedulerApp.java:136)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.assignContainer(FifoScheduler.java:683)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.assignNodeLocalContainers(FifoScheduler.java:602)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.assignContainersOnNode(FifoScheduler.java:560)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.assignContainers(FifoScheduler.java:488)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.nodeUpdate(FifoScheduler.java:729)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.handle(FifoScheduler.java:774)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.handle(FifoScheduler.java:101)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher$EventProcessor.run(ResourceManager.java:599)
at java.lang.Thread.run(Thread.java:662)
2014-07-31 04:18:19,655 INFO
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Exiting, bbye..
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (YARN-1572) Low chance to hit NPE issue in AppSchedulingInfo#allocateNodeLocal
[
https://issues.apache.org/jira/browse/YARN-1572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kareem El Gebaly updated YARN-1572:
---
Release Note: (was: Fix for YARN-1572 added if !null guarding condition)
Low chance to hit NPE issue in AppSchedulingInfo#allocateNodeLocal
--
Key: YARN-1572
URL: https://issues.apache.org/jira/browse/YARN-1572
Project: Hadoop YARN
Issue Type: Bug
Components: scheduler
Affects Versions: 2.3.0
Reporter: Wenwu Peng
Assignee: Wenwu Peng
Attachments: YARN-1572-log.tar.gz, conf.tar.gz, log.tar.gz
we have lower chance to hit NPE in allocateNodeLocal when run benchmark(hit
4 in 20 times).
{code}
2014-07-31 04:18:19,653 INFO
org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerNode:
Assigned container container_1406794589275_0001_01_21 of capacity
memory:1024, vCores:1 on host datanode10:57281, which has 6 containers,
memory:6144, vCores:6 used and memory:2048, vCores:2 available after
allocation
2014-07-31 04:18:19,654 FATAL
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error in
handling event type NODE_UPDATE to the scheduler
java.lang.NullPointerException
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.AppSchedulingInfo.allocateNodeLocal(AppSchedulingInfo.java:311)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.AppSchedulingInfo.allocate(AppSchedulingInfo.java:268)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.common.fica.FiCaSchedulerApp.allocate(FiCaSchedulerApp.java:136)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.assignContainer(FifoScheduler.java:683)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.assignNodeLocalContainers(FifoScheduler.java:602)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.assignContainersOnNode(FifoScheduler.java:560)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.assignContainers(FifoScheduler.java:488)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.nodeUpdate(FifoScheduler.java:729)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.handle(FifoScheduler.java:774)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.handle(FifoScheduler.java:101)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher$EventProcessor.run(ResourceManager.java:599)
at java.lang.Thread.run(Thread.java:662)
2014-07-31 04:18:19,655 INFO
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Exiting, bbye..
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (YARN-1572) Low chance to hit NPE issue in AppSchedulingInfo#allocateNodeLocal
[
https://issues.apache.org/jira/browse/YARN-1572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
yangping wu updated YARN-1572:
--
Description:
we have lower chance to hit NPE in allocateNodeLocal when run benchmark(hit 4
in 20 times).
{code}
2014-07-31 04:18:19,653 INFO
org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerNode: Assigned
container container_1406794589275_0001_01_21 of capacity memory:1024,
vCores:1 on host datanode10:57281, which has 6 containers, memory:6144,
vCores:6 used and memory:2048, vCores:2 available after allocation
2014-07-31 04:18:19,654 FATAL
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error in
handling event type NODE_UPDATE to the scheduler
java.lang.NullPointerException
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.AppSchedulingInfo.allocateNodeLocal(AppSchedulingInfo.java:311)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.AppSchedulingInfo.allocate(AppSchedulingInfo.java:268)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.common.fica.FiCaSchedulerApp.allocate(FiCaSchedulerApp.java:136)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.assignContainer(FifoScheduler.java:683)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.assignNodeLocalContainers(FifoScheduler.java:602)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.assignContainersOnNode(FifoScheduler.java:560)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.assignContainers(FifoScheduler.java:488)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.nodeUpdate(FifoScheduler.java:729)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.handle(FifoScheduler.java:774)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.handle(FifoScheduler.java:101)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher$EventProcessor.run(ResourceManager.java:599)
at java.lang.Thread.run(Thread.java:662)
2014-07-31 04:18:19,655 INFO
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Exiting, bbye..
{code}
was:
we have lower chance to hit NPE in allocateNodeLocal when run benchmark(hit 4
in 20 times).
2014-07-31 04:18:19,653 INFO
org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerNode: Assigned
container container_1406794589275_0001_01_21 of capacity memory:1024,
vCores:1 on host datanode10:57281, which has 6 containers, memory:6144,
vCores:6 used and memory:2048, vCores:2 available after allocation
2014-07-31 04:18:19,654 FATAL
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error in
handling event type NODE_UPDATE to the scheduler
java.lang.NullPointerException
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.AppSchedulingInfo.allocateNodeLocal(AppSchedulingInfo.java:311)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.AppSchedulingInfo.allocate(AppSchedulingInfo.java:268)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.common.fica.FiCaSchedulerApp.allocate(FiCaSchedulerApp.java:136)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.assignContainer(FifoScheduler.java:683)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.assignNodeLocalContainers(FifoScheduler.java:602)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.assignContainersOnNode(FifoScheduler.java:560)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.assignContainers(FifoScheduler.java:488)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.nodeUpdate(FifoScheduler.java:729)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.handle(FifoScheduler.java:774)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.handle(FifoScheduler.java:101)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher$EventProcessor.run(ResourceManager.java:599)
at java.lang.Thread.run(Thread.java:662)
2014-07-31 04:18:19,655 INFO
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Exiting, bbye..
Low chance to hit NPE issue in AppSchedulingInfo#allocateNodeLocal
--
Key: YARN-1572
URL: https://issues.apache.org/jira/browse/YARN-1572
Project: Hadoop YARN
Issue Type: Bug
Components: scheduler
Affects Versions: 2.2.0
Reporter: Wenwu Peng
[jira] [Updated] (YARN-1572) Low chance to hit NPE issue in AppSchedulingInfo#allocateNodeLocal
[
https://issues.apache.org/jira/browse/YARN-1572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wenwu Peng updated YARN-1572:
-
Attachment: YARN-1572-log.tar.gz
Thanks a lot Junping! please refer to YARN-1572-log.tar.gz for the log of NPE
for latest trunk.
java.lang.NullPointerException
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.AppSchedulingInfo.allocateNodeLocal(AppSchedulingInfo.java:311)
Low chance to hit NPE issue in AppSchedulingInfo#allocateNodeLocal
--
Key: YARN-1572
URL: https://issues.apache.org/jira/browse/YARN-1572
Project: Hadoop YARN
Issue Type: Bug
Components: scheduler
Affects Versions: 2.2.0
Reporter: Wenwu Peng
Assignee: Wenwu Peng
Attachments: YARN-1572-log.tar.gz, conf.tar.gz, log.tar.gz
we have lower chance to hit NPE in allocateNodeLocal when run benchmark(hit
4 in 20 times).
Steps:
1. setup hadoop 2.2.0 environment
2. Run for i in {1..10}; do /hadoop/hadoop-smoke/bin/hadoop jar
/hadoop/hadoop-smoke/share/hadoop/mapreduce/hadoop-mapreduce-client-common-*.jar
org.apache.hadoop.fs.TestDFSIO -write -nrFiles 30 -fileSize 64MB; sleep
10;done
2014-01-08 03:56:14,082 FATAL
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error in
handling event type NODE_UPDATE to the scheduler
java.lang.NullPointerException
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.AppSchedulingInfo.allocateNodeLocal(AppSchedulingInfo.java:291)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.AppSchedulingInfo.allocate(AppSchedulingInfo.java:252)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.common.fica.FiCaSchedulerApp.allocate(FiCaSchedulerApp.java:294)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.assignContainer(FifoScheduler.java:614)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.assignNodeLocalContainers(FifoScheduler.java:524)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.assignContainersOnNode(FifoScheduler.java:482)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.assignContainers(FifoScheduler.java:419)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.nodeUpdate(FifoScheduler.java:658)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.handle(FifoScheduler.java:687)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.handle(FifoScheduler.java:95)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher$EventProcessor.run(ResourceManager.java:440)
at java.lang.Thread.run(Thread.java:662)
will attach log and configure files later
Note:
My topology file:
10.111.89.230 /QE1/sin2-pekaurora-bdcqe046.eng.vmware.com
10.111.89.231 /QE1/sin2-pekaurora-bdcqe046.eng.vmware.com
10.111.89.232 /QE1/sin2-pekaurora-bdcqe046.eng.vmware.com
10.111.89.239 /QE1/sin2-pekaurora-bdcqe046.eng.vmware.com
10.111.89.233 /QE1/sin2-pekaurora-bdcqe017.eng.vmware.com
10.111.89.234 /QE1/sin2-pekaurora-bdcqe017.eng.vmware.com
10.111.89.240 /QE1/sin2-pekaurora-bdcqe017.eng.vmware.com
10.111.89.236 /QE2/sin2-pekaurora-bdcqe047.eng.vmware.com
10.111.89.241 /QE2/sin2-pekaurora-bdcqe047.eng.vmware.com
10.111.89.238 /QE2/sin2-pekaurora-bdcqe048.eng.vmware.com
10.111.89.242 /QE2/sin2-pekaurora-bdcqe048.eng.vmware.com
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (YARN-1572) Low chance to hit NPE issue in AppSchedulingInfo#allocateNodeLocal
[
https://issues.apache.org/jira/browse/YARN-1572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wenwu Peng updated YARN-1572:
-
Description:
we have lower chance to hit NPE in allocateNodeLocal when run benchmark(hit 4
in 20 times).
2014-07-31 04:18:19,653 INFO
org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerNode: Assigned
container container_1406794589275_0001_01_21 of capacity memory:1024,
vCores:1 on host datanode10:57281, which has 6 containers, memory:6144,
vCores:6 used and memory:2048, vCores:2 available after allocation
2014-07-31 04:18:19,654 FATAL
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error in
handling event type NODE_UPDATE to the scheduler
java.lang.NullPointerException
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.AppSchedulingInfo.allocateNodeLocal(AppSchedulingInfo.java:311)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.AppSchedulingInfo.allocate(AppSchedulingInfo.java:268)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.common.fica.FiCaSchedulerApp.allocate(FiCaSchedulerApp.java:136)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.assignContainer(FifoScheduler.java:683)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.assignNodeLocalContainers(FifoScheduler.java:602)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.assignContainersOnNode(FifoScheduler.java:560)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.assignContainers(FifoScheduler.java:488)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.nodeUpdate(FifoScheduler.java:729)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.handle(FifoScheduler.java:774)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.handle(FifoScheduler.java:101)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher$EventProcessor.run(ResourceManager.java:599)
at java.lang.Thread.run(Thread.java:662)
2014-07-31 04:18:19,655 INFO
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Exiting, bbye..
was:
we have lower chance to hit NPE in allocateNodeLocal when run benchmark(hit 4
in 20 times).
Steps:
1. setup hadoop 2.2.0 environment
2. Run for i in {1..10}; do /hadoop/hadoop-smoke/bin/hadoop jar
/hadoop/hadoop-smoke/share/hadoop/mapreduce/hadoop-mapreduce-client-common-*.jar
org.apache.hadoop.fs.TestDFSIO -write -nrFiles 30 -fileSize 64MB; sleep 10;done
2014-01-08 03:56:14,082 FATAL
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error in
handling event type NODE_UPDATE to the scheduler
java.lang.NullPointerException
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.AppSchedulingInfo.allocateNodeLocal(AppSchedulingInfo.java:291)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.AppSchedulingInfo.allocate(AppSchedulingInfo.java:252)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.common.fica.FiCaSchedulerApp.allocate(FiCaSchedulerApp.java:294)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.assignContainer(FifoScheduler.java:614)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.assignNodeLocalContainers(FifoScheduler.java:524)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.assignContainersOnNode(FifoScheduler.java:482)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.assignContainers(FifoScheduler.java:419)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.nodeUpdate(FifoScheduler.java:658)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.handle(FifoScheduler.java:687)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.handle(FifoScheduler.java:95)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher$EventProcessor.run(ResourceManager.java:440)
at java.lang.Thread.run(Thread.java:662)
will attach log and configure files later
Note:
My topology file:
10.111.89.230 /QE1/sin2-pekaurora-bdcqe046.eng.vmware.com
10.111.89.231 /QE1/sin2-pekaurora-bdcqe046.eng.vmware.com
10.111.89.232 /QE1/sin2-pekaurora-bdcqe046.eng.vmware.com
10.111.89.239 /QE1/sin2-pekaurora-bdcqe046.eng.vmware.com
10.111.89.233 /QE1/sin2-pekaurora-bdcqe017.eng.vmware.com
10.111.89.234 /QE1/sin2-pekaurora-bdcqe017.eng.vmware.com
10.111.89.240 /QE1/sin2-pekaurora-bdcqe017.eng.vmware.com
10.111.89.236 /QE2/sin2-pekaurora-bdcqe047.eng.vmware.com
10.111.89.241
[jira] [Updated] (YARN-1572) Low chance to hit NPE issue in AppSchedulingInfo#allocateNodeLocal
[
https://issues.apache.org/jira/browse/YARN-1572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Junping Du updated YARN-1572:
-
Summary: Low chance to hit NPE issue in AppSchedulingInfo#allocateNodeLocal
(was: Lower chance to hit NPE in allocateNodeLocal )
Low chance to hit NPE issue in AppSchedulingInfo#allocateNodeLocal
--
Key: YARN-1572
URL: https://issues.apache.org/jira/browse/YARN-1572
Project: Hadoop YARN
Issue Type: Bug
Affects Versions: 2.2.0
Reporter: wenwupeng
Assignee: Junping Du
Attachments: conf.tar.gz, log.tar.gz
we have lower chance to hit NPE in allocateNodeLocal when run benchmark(hit
4 in 20 times).
Steps:
1. setup hadoop 2.2.0 environment
2. Run for i in {1..10}; do /hadoop/hadoop-smoke/bin/hadoop jar
/hadoop/hadoop-smoke/share/hadoop/mapreduce/hadoop-mapreduce-client-common-*.jar
org.apache.hadoop.fs.TestDFSIO -write -nrFiles 30 -fileSize 64MB; sleep
10;done
2014-01-08 03:56:14,082 FATAL
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error in
handling event type NODE_UPDATE to the scheduler
java.lang.NullPointerException
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.AppSchedulingInfo.allocateNodeLocal(AppSchedulingInfo.java:291)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.AppSchedulingInfo.allocate(AppSchedulingInfo.java:252)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.common.fica.FiCaSchedulerApp.allocate(FiCaSchedulerApp.java:294)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.assignContainer(FifoScheduler.java:614)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.assignNodeLocalContainers(FifoScheduler.java:524)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.assignContainersOnNode(FifoScheduler.java:482)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.assignContainers(FifoScheduler.java:419)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.nodeUpdate(FifoScheduler.java:658)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.handle(FifoScheduler.java:687)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.handle(FifoScheduler.java:95)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher$EventProcessor.run(ResourceManager.java:440)
at java.lang.Thread.run(Thread.java:662)
will attach log and configure files later
Note:
My topology file:
10.111.89.230 /QE1/sin2-pekaurora-bdcqe046.eng.vmware.com
10.111.89.231 /QE1/sin2-pekaurora-bdcqe046.eng.vmware.com
10.111.89.232 /QE1/sin2-pekaurora-bdcqe046.eng.vmware.com
10.111.89.239 /QE1/sin2-pekaurora-bdcqe046.eng.vmware.com
10.111.89.233 /QE1/sin2-pekaurora-bdcqe017.eng.vmware.com
10.111.89.234 /QE1/sin2-pekaurora-bdcqe017.eng.vmware.com
10.111.89.240 /QE1/sin2-pekaurora-bdcqe017.eng.vmware.com
10.111.89.236 /QE2/sin2-pekaurora-bdcqe047.eng.vmware.com
10.111.89.241 /QE2/sin2-pekaurora-bdcqe047.eng.vmware.com
10.111.89.238 /QE2/sin2-pekaurora-bdcqe048.eng.vmware.com
10.111.89.242 /QE2/sin2-pekaurora-bdcqe048.eng.vmware.com
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (YARN-1572) Low chance to hit NPE issue in AppSchedulingInfo#allocateNodeLocal
[
https://issues.apache.org/jira/browse/YARN-1572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Junping Du updated YARN-1572:
-
Affects Version/s: 2.2.0
Low chance to hit NPE issue in AppSchedulingInfo#allocateNodeLocal
--
Key: YARN-1572
URL: https://issues.apache.org/jira/browse/YARN-1572
Project: Hadoop YARN
Issue Type: Bug
Affects Versions: 2.2.0
Reporter: wenwupeng
Assignee: Junping Du
Attachments: conf.tar.gz, log.tar.gz
we have lower chance to hit NPE in allocateNodeLocal when run benchmark(hit
4 in 20 times).
Steps:
1. setup hadoop 2.2.0 environment
2. Run for i in {1..10}; do /hadoop/hadoop-smoke/bin/hadoop jar
/hadoop/hadoop-smoke/share/hadoop/mapreduce/hadoop-mapreduce-client-common-*.jar
org.apache.hadoop.fs.TestDFSIO -write -nrFiles 30 -fileSize 64MB; sleep
10;done
2014-01-08 03:56:14,082 FATAL
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error in
handling event type NODE_UPDATE to the scheduler
java.lang.NullPointerException
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.AppSchedulingInfo.allocateNodeLocal(AppSchedulingInfo.java:291)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.AppSchedulingInfo.allocate(AppSchedulingInfo.java:252)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.common.fica.FiCaSchedulerApp.allocate(FiCaSchedulerApp.java:294)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.assignContainer(FifoScheduler.java:614)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.assignNodeLocalContainers(FifoScheduler.java:524)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.assignContainersOnNode(FifoScheduler.java:482)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.assignContainers(FifoScheduler.java:419)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.nodeUpdate(FifoScheduler.java:658)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.handle(FifoScheduler.java:687)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.handle(FifoScheduler.java:95)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher$EventProcessor.run(ResourceManager.java:440)
at java.lang.Thread.run(Thread.java:662)
will attach log and configure files later
Note:
My topology file:
10.111.89.230 /QE1/sin2-pekaurora-bdcqe046.eng.vmware.com
10.111.89.231 /QE1/sin2-pekaurora-bdcqe046.eng.vmware.com
10.111.89.232 /QE1/sin2-pekaurora-bdcqe046.eng.vmware.com
10.111.89.239 /QE1/sin2-pekaurora-bdcqe046.eng.vmware.com
10.111.89.233 /QE1/sin2-pekaurora-bdcqe017.eng.vmware.com
10.111.89.234 /QE1/sin2-pekaurora-bdcqe017.eng.vmware.com
10.111.89.240 /QE1/sin2-pekaurora-bdcqe017.eng.vmware.com
10.111.89.236 /QE2/sin2-pekaurora-bdcqe047.eng.vmware.com
10.111.89.241 /QE2/sin2-pekaurora-bdcqe047.eng.vmware.com
10.111.89.238 /QE2/sin2-pekaurora-bdcqe048.eng.vmware.com
10.111.89.242 /QE2/sin2-pekaurora-bdcqe048.eng.vmware.com
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (YARN-1572) Low chance to hit NPE issue in AppSchedulingInfo#allocateNodeLocal
[
https://issues.apache.org/jira/browse/YARN-1572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Junping Du updated YARN-1572:
-
Component/s: scheduler
Low chance to hit NPE issue in AppSchedulingInfo#allocateNodeLocal
--
Key: YARN-1572
URL: https://issues.apache.org/jira/browse/YARN-1572
Project: Hadoop YARN
Issue Type: Bug
Components: scheduler
Affects Versions: 2.2.0
Reporter: wenwupeng
Assignee: Junping Du
Attachments: conf.tar.gz, log.tar.gz
we have lower chance to hit NPE in allocateNodeLocal when run benchmark(hit
4 in 20 times).
Steps:
1. setup hadoop 2.2.0 environment
2. Run for i in {1..10}; do /hadoop/hadoop-smoke/bin/hadoop jar
/hadoop/hadoop-smoke/share/hadoop/mapreduce/hadoop-mapreduce-client-common-*.jar
org.apache.hadoop.fs.TestDFSIO -write -nrFiles 30 -fileSize 64MB; sleep
10;done
2014-01-08 03:56:14,082 FATAL
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error in
handling event type NODE_UPDATE to the scheduler
java.lang.NullPointerException
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.AppSchedulingInfo.allocateNodeLocal(AppSchedulingInfo.java:291)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.AppSchedulingInfo.allocate(AppSchedulingInfo.java:252)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.common.fica.FiCaSchedulerApp.allocate(FiCaSchedulerApp.java:294)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.assignContainer(FifoScheduler.java:614)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.assignNodeLocalContainers(FifoScheduler.java:524)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.assignContainersOnNode(FifoScheduler.java:482)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.assignContainers(FifoScheduler.java:419)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.nodeUpdate(FifoScheduler.java:658)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.handle(FifoScheduler.java:687)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.handle(FifoScheduler.java:95)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher$EventProcessor.run(ResourceManager.java:440)
at java.lang.Thread.run(Thread.java:662)
will attach log and configure files later
Note:
My topology file:
10.111.89.230 /QE1/sin2-pekaurora-bdcqe046.eng.vmware.com
10.111.89.231 /QE1/sin2-pekaurora-bdcqe046.eng.vmware.com
10.111.89.232 /QE1/sin2-pekaurora-bdcqe046.eng.vmware.com
10.111.89.239 /QE1/sin2-pekaurora-bdcqe046.eng.vmware.com
10.111.89.233 /QE1/sin2-pekaurora-bdcqe017.eng.vmware.com
10.111.89.234 /QE1/sin2-pekaurora-bdcqe017.eng.vmware.com
10.111.89.240 /QE1/sin2-pekaurora-bdcqe017.eng.vmware.com
10.111.89.236 /QE2/sin2-pekaurora-bdcqe047.eng.vmware.com
10.111.89.241 /QE2/sin2-pekaurora-bdcqe047.eng.vmware.com
10.111.89.238 /QE2/sin2-pekaurora-bdcqe048.eng.vmware.com
10.111.89.242 /QE2/sin2-pekaurora-bdcqe048.eng.vmware.com
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (YARN-1572) Low chance to hit NPE issue in AppSchedulingInfo#allocateNodeLocal
[
https://issues.apache.org/jira/browse/YARN-1572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Junping Du updated YARN-1572:
-
Target Version/s: 2.4.0
Low chance to hit NPE issue in AppSchedulingInfo#allocateNodeLocal
--
Key: YARN-1572
URL: https://issues.apache.org/jira/browse/YARN-1572
Project: Hadoop YARN
Issue Type: Bug
Components: scheduler
Affects Versions: 2.2.0
Reporter: wenwupeng
Assignee: Junping Du
Attachments: conf.tar.gz, log.tar.gz
we have lower chance to hit NPE in allocateNodeLocal when run benchmark(hit
4 in 20 times).
Steps:
1. setup hadoop 2.2.0 environment
2. Run for i in {1..10}; do /hadoop/hadoop-smoke/bin/hadoop jar
/hadoop/hadoop-smoke/share/hadoop/mapreduce/hadoop-mapreduce-client-common-*.jar
org.apache.hadoop.fs.TestDFSIO -write -nrFiles 30 -fileSize 64MB; sleep
10;done
2014-01-08 03:56:14,082 FATAL
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error in
handling event type NODE_UPDATE to the scheduler
java.lang.NullPointerException
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.AppSchedulingInfo.allocateNodeLocal(AppSchedulingInfo.java:291)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.AppSchedulingInfo.allocate(AppSchedulingInfo.java:252)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.common.fica.FiCaSchedulerApp.allocate(FiCaSchedulerApp.java:294)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.assignContainer(FifoScheduler.java:614)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.assignNodeLocalContainers(FifoScheduler.java:524)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.assignContainersOnNode(FifoScheduler.java:482)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.assignContainers(FifoScheduler.java:419)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.nodeUpdate(FifoScheduler.java:658)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.handle(FifoScheduler.java:687)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.handle(FifoScheduler.java:95)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher$EventProcessor.run(ResourceManager.java:440)
at java.lang.Thread.run(Thread.java:662)
will attach log and configure files later
Note:
My topology file:
10.111.89.230 /QE1/sin2-pekaurora-bdcqe046.eng.vmware.com
10.111.89.231 /QE1/sin2-pekaurora-bdcqe046.eng.vmware.com
10.111.89.232 /QE1/sin2-pekaurora-bdcqe046.eng.vmware.com
10.111.89.239 /QE1/sin2-pekaurora-bdcqe046.eng.vmware.com
10.111.89.233 /QE1/sin2-pekaurora-bdcqe017.eng.vmware.com
10.111.89.234 /QE1/sin2-pekaurora-bdcqe017.eng.vmware.com
10.111.89.240 /QE1/sin2-pekaurora-bdcqe017.eng.vmware.com
10.111.89.236 /QE2/sin2-pekaurora-bdcqe047.eng.vmware.com
10.111.89.241 /QE2/sin2-pekaurora-bdcqe047.eng.vmware.com
10.111.89.238 /QE2/sin2-pekaurora-bdcqe048.eng.vmware.com
10.111.89.242 /QE2/sin2-pekaurora-bdcqe048.eng.vmware.com
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
