[
https://issues.apache.org/jira/browse/YARN-5210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Varun Saxena updated YARN-5210:
-------------------------------
Description:
Found a couple of issues while testing ATSv2.
* There is a NPE while publishing DS_CONTAINER_START_EVENT which means that
this event is not published.
{noformat}
2016-06-07 23:19:00,020
[org.apache.hadoop.yarn.client.api.async.impl.NMClientAsyncImpl #0] INFO
org.apache.hadoop.yarn.client.api.async.impl.NMClientAsyncImpl: Unchecked
exception is thrown from onContainerStarted for Container
container_e77_1465311876353_0007_01_000002
java.lang.NullPointerException
at
org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl.putEntities(TimelineClientImpl.java:389)
at
org.apache.hadoop.yarn.applications.distributedshell.ApplicationMaster.putContainerEntity(ApplicationMaster.java:1284)
at
org.apache.hadoop.yarn.applications.distributedshell.ApplicationMaster.publishContainerStartEvent(ApplicationMaster.java:1235)
at
org.apache.hadoop.yarn.applications.distributedshell.ApplicationMaster.access$1200(ApplicationMaster.java:175)
at
org.apache.hadoop.yarn.applications.distributedshell.ApplicationMaster$NMCallbackHandler.onContainerStarted(ApplicationMaster.java:986)
at
org.apache.hadoop.yarn.client.api.async.impl.NMClientAsyncImpl$StatefulContainer$StartContainerTransition.transition(NMClientAsyncImpl.java:454)
at
org.apache.hadoop.yarn.client.api.async.impl.NMClientAsyncImpl$StatefulContainer$StartContainerTransition.transition(NMClientAsyncImpl.java:436)
at
org.apache.hadoop.yarn.state.StateMachineFactory$MultipleInternalArc.doTransition(StateMachineFactory.java:385)
at
org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:302)
at
org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46)
at
org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448)
at
org.apache.hadoop.yarn.client.api.async.impl.NMClientAsyncImpl$StatefulContainer.handle(NMClientAsyncImpl.java:617)
at
org.apache.hadoop.yarn.client.api.async.impl.NMClientAsyncImpl$ContainerEventProcessor.run(NMClientAsyncImpl.java:676)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{noformat}
* Created time is not reported from distributed shell for both DS_CONTAINER and
DS_APP_ATTEMPT entities.
As can be seen below, when we query DS_APP_ATTEMPT entities, we do not get
createdtime in response.
{code}
[
{
"metrics": [ ],
"events": [ ],
"type": "DS_APP_ATTEMPT",
"id": "appattempt_1465246237936_0003_000001",
"isrelatedto": { },
"relatesto": { },
"info": {
"UID":
"yarn-cluster!application_1465246237936_0003!DS_APP_ATTEMPT!appattempt_1465246237936_0003_000001"
},
"configs": { }
}
]
{code}
As can be seen from response received upon querying a DS_CONTAINER entity we
can see that createdtime is not present and DS_CONTAINER_START is not present
either(due to NPE pointed above).
{code}
{
"metrics": [ ],
"events": [
{
"id": "DS_CONTAINER_END",
"timestamp": 1465314587480,
"info": {
"Exit Status": 0,
"State": "COMPLETE"
}
}
],
"type": "DS_CONTAINER",
"id": "container_e77_1465311876353_0003_01_000002",
"isrelatedto": { },
"relatesto": { },
"info": {
"UID":
"yarn-cluster!application_1465311876353_0003!DS_CONTAINER!container_e77_1465311876353_0003_01_000002"
},
"configs": { }
}
{code}
was:
Found a couple of issues while testing ATSv2.
* There is a NPE while publishing DS_CONTAINER_START_EVENT which means that
this event is not published.
{noformat}
2016-06-07 23:19:00,020
[org.apache.hadoop.yarn.client.api.async.impl.NMClientAsyncImpl #0] INFO
org.apache.hadoop.yarn.client.api.async.impl.NMClientAsyncImpl: Unchecked
exception is thrown from onContainerStarted for Container
container_e77_1465311876353_0007_01_000002
java.lang.NullPointerException
at
org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl.putEntities(TimelineClientImpl.java:389)
at
org.apache.hadoop.yarn.applications.distributedshell.ApplicationMaster.putContainerEntity(ApplicationMaster.java:1284)
at
org.apache.hadoop.yarn.applications.distributedshell.ApplicationMaster.publishContainerStartEvent(ApplicationMaster.java:1235)
at
org.apache.hadoop.yarn.applications.distributedshell.ApplicationMaster.access$1200(ApplicationMaster.java:175)
at
org.apache.hadoop.yarn.applications.distributedshell.ApplicationMaster$NMCallbackHandler.onContainerStarted(ApplicationMaster.java:986)
at
org.apache.hadoop.yarn.client.api.async.impl.NMClientAsyncImpl$StatefulContainer$StartContainerTransition.transition(NMClientAsyncImpl.java:454)
at
org.apache.hadoop.yarn.client.api.async.impl.NMClientAsyncImpl$StatefulContainer$StartContainerTransition.transition(NMClientAsyncImpl.java:436)
at
org.apache.hadoop.yarn.state.StateMachineFactory$MultipleInternalArc.doTransition(StateMachineFactory.java:385)
at
org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:302)
at
org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46)
at
org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448)
at
org.apache.hadoop.yarn.client.api.async.impl.NMClientAsyncImpl$StatefulContainer.handle(NMClientAsyncImpl.java:617)
at
org.apache.hadoop.yarn.client.api.async.impl.NMClientAsyncImpl$ContainerEventProcessor.run(NMClientAsyncImpl.java:676)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{noformat}
* Created time is not reported from distributed shell for both DS_CONTAINER and
DS_APP_ATTEMPT entities.
As can be seen below, when we query DS_APP_ATTEMPT entities, we do not get
createdtime in response.
{code}
[
{
"metrics": [ ],
"events": [ ],
"type": "DS_APP_ATTEMPT",
"id": "appattempt_1465246237936_0003_000001",
"isrelatedto": { },
"relatesto": { },
"info": {
"UID":
"yarn-cluster!application_1465246237936_0003!DS_APP_ATTEMPT!appattempt_1465246237936_0003_000001"
},
"configs": { }
}
]
{code}
As can be seen from response received upon querying a DS_CONTAINER entity we
can see that createdtime is not present and DS_CONTAINER_START is not present
either(due to NPE pointed above).
{code}
{
"metrics": [ ],
"events": [
{
"id": "DS_CONTAINER_END",
"timestamp": 1465314587480,
"info": {
"Exit Status": 0,
"State": "COMPLETE"
}
}
],
"type": "DS_CONTAINER",
"id": "container_e77_1465311876353_0003_01_000002",
"isrelatedto": { },
"relatesto": { },
"info": {
"UID":
"yarn-cluster!application_1465311876353_0003!DS_CONTAINER!container_e77_1465311876353_0003_01_000002"
},
"configs": { }
}
{code}
> NPE in Distributed Shell while publishing DS_CONTAINER_START event and other
> miscellaneous issues
> -------------------------------------------------------------------------------------------------
>
> Key: YARN-5210
> URL: https://issues.apache.org/jira/browse/YARN-5210
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
> Affects Versions: YARN-2928
> Reporter: Varun Saxena
> Assignee: Varun Saxena
> Labels: yarn-2928-1st-milestone
>
> Found a couple of issues while testing ATSv2.
> * There is a NPE while publishing DS_CONTAINER_START_EVENT which means that
> this event is not published.
> {noformat}
> 2016-06-07 23:19:00,020
> [org.apache.hadoop.yarn.client.api.async.impl.NMClientAsyncImpl #0] INFO
> org.apache.hadoop.yarn.client.api.async.impl.NMClientAsyncImpl: Unchecked
> exception is thrown from onContainerStarted for Container
> container_e77_1465311876353_0007_01_000002
> java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl.putEntities(TimelineClientImpl.java:389)
> at
> org.apache.hadoop.yarn.applications.distributedshell.ApplicationMaster.putContainerEntity(ApplicationMaster.java:1284)
> at
> org.apache.hadoop.yarn.applications.distributedshell.ApplicationMaster.publishContainerStartEvent(ApplicationMaster.java:1235)
> at
> org.apache.hadoop.yarn.applications.distributedshell.ApplicationMaster.access$1200(ApplicationMaster.java:175)
> at
> org.apache.hadoop.yarn.applications.distributedshell.ApplicationMaster$NMCallbackHandler.onContainerStarted(ApplicationMaster.java:986)
> at
> org.apache.hadoop.yarn.client.api.async.impl.NMClientAsyncImpl$StatefulContainer$StartContainerTransition.transition(NMClientAsyncImpl.java:454)
> at
> org.apache.hadoop.yarn.client.api.async.impl.NMClientAsyncImpl$StatefulContainer$StartContainerTransition.transition(NMClientAsyncImpl.java:436)
> at
> org.apache.hadoop.yarn.state.StateMachineFactory$MultipleInternalArc.doTransition(StateMachineFactory.java:385)
> at
> org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:302)
> at
> org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46)
> at
> org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448)
> at
> org.apache.hadoop.yarn.client.api.async.impl.NMClientAsyncImpl$StatefulContainer.handle(NMClientAsyncImpl.java:617)
> at
> org.apache.hadoop.yarn.client.api.async.impl.NMClientAsyncImpl$ContainerEventProcessor.run(NMClientAsyncImpl.java:676)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
> * Created time is not reported from distributed shell for both DS_CONTAINER
> and DS_APP_ATTEMPT entities.
> As can be seen below, when we query DS_APP_ATTEMPT entities, we do not get
> createdtime in response.
> {code}
> [
> {
> "metrics": [ ],
> "events": [ ],
> "type": "DS_APP_ATTEMPT",
> "id": "appattempt_1465246237936_0003_000001",
> "isrelatedto": { },
> "relatesto": { },
> "info": {
> "UID":
> "yarn-cluster!application_1465246237936_0003!DS_APP_ATTEMPT!appattempt_1465246237936_0003_000001"
> },
> "configs": { }
> }
> ]
> {code}
> As can be seen from response received upon querying a DS_CONTAINER entity we
> can see that createdtime is not present and DS_CONTAINER_START is not present
> either(due to NPE pointed above).
> {code}
> {
> "metrics": [ ],
> "events": [
> {
> "id": "DS_CONTAINER_END",
> "timestamp": 1465314587480,
> "info": {
> "Exit Status": 0,
> "State": "COMPLETE"
> }
> }
> ],
> "type": "DS_CONTAINER",
> "id": "container_e77_1465311876353_0003_01_000002",
> "isrelatedto": { },
> "relatesto": { },
> "info": {
> "UID":
> "yarn-cluster!application_1465311876353_0003!DS_CONTAINER!container_e77_1465311876353_0003_01_000002"
> },
> "configs": { }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
