[
https://issues.apache.org/jira/browse/YARN-9418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16804054#comment-16804054
]
Prabhu Joseph commented on YARN-9418:
-------------------------------------
Have used inverted ContainerId#getContainerId as entityIdPrefix for
YARN_CONTAINER entities for both NM and RM. TimelineReader fetches
ContainerEntity from both NM and RM.
{code:java}
[hbase@yarn-ats-3 BACKUP]$ curl -s
"http://yarn-ats-3:8198/ws/v2/timeline/apps/application_1553788280931_0001/entities/YARN_CONTAINER/container_e30_1553788280931_0001_01_000001?user.name=hbase&fields=ALL";
| jq .
{
"metrics": [
{
"type": "SINGLE_VALUE",
"id": "CPU",
"aggregationOp": "NOP",
"values": {
"1553788338798": 2
}
},
{
"type": "SINGLE_VALUE",
"id": "MEMORY",
"aggregationOp": "NOP",
"values": {
"1553788338798": 488013824
}
}
],
"events": [
{
"id": "YARN_RM_CONTAINER_FINISHED",
"timestamp": 1553788341158,
"info": {}
},
{
"id": "YARN_CONTAINER_FINISHED",
"timestamp": 1553788341151,
"info": {}
},
{
"id": "YARN_NM_CONTAINER_LOCALIZATION_FINISHED",
"timestamp": 1553788316027,
"info": {}
},
{
"id": "YARN_NM_CONTAINER_LOCALIZATION_STARTED",
"timestamp": 1553788315294,
"info": {}
},
{
"id": "YARN_CONTAINER_CREATED",
"timestamp": 1553788315284,
"info": {}
},
{
"id": "YARN_RM_CONTAINER_CREATED",
"timestamp": 1553788314339,
"info": {}
}
],
"createdtime": 1553788314809,
"idprefix": 9223339051505943000,
"info": {
"YARN_CONTAINER_STATE": "COMPLETE",
"YARN_CONTAINER_ALLOCATED_HOST": "yarn-ats-1",
"YARN_CONTAINER_ALLOCATED_HOST_HTTP_ADDRESS": "yarn-ats-1:8042",
"YARN_CONTAINER_ALLOCATED_VCORE": 1,
"FROM_ID": "ats!hbase!word
count!1553788313450!application_1553788280931_0001!YARN_CONTAINER!9223339051505942526!container_e30_1553788280931_0001_01_000001",
"YARN_CONTAINER_ALLOCATED_PORT": 45454,
"UID":
"ats!application_1553788280931_0001!YARN_CONTAINER!9223339051505942526!container_e30_1553788280931_0001_01_000001",
"YARN_CONTAINER_ALLOCATED_MEMORY": 2048,
"SYSTEM_INFO_PARENT_ENTITY": {
"type": "YARN_APPLICATION_ATTEMPT",
"id": "appattempt_1553788280931_0001_000001"
},
"YARN_CONTAINER_EXIT_STATUS": 0,
"YARN_CONTAINER_ALLOCATED_PRIORITY": "0",
"YARN_CONTAINER_DIAGNOSTICS_INFO": "",
"YARN_CONTAINER_FINISHED_TIME": 1553788341151
},
"configs": {},
"isrelatedto": {},
"relatesto": {},
"id": "container_e30_1553788280931_0001_01_000001",
"type": "YARN_CONTAINER"
}{code}
> ATSV2 /apps/${appId}/entities/YARN_CONTAINER rest api does not show metrics
> ---------------------------------------------------------------------------
>
> Key: YARN-9418
> URL: https://issues.apache.org/jira/browse/YARN-9418
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: ATSv2
> Affects Versions: 3.2.0
> Reporter: Prabhu Joseph
> Assignee: Prabhu Joseph
> Priority: Critical
> Attachments: YARN-9418-001.patch
>
>
> ATSV2 entities rest api does not show the metrics
> {code:java}
> [hbase@yarn-ats-3 centos]$ curl -s
> "http://yarn-ats-3:8198/ws/v2/timeline/apps/application_1553685341603_0006/entities/YARN_CONTAINER/container_e18_1553685341603_0006_01_000001?user.name=hbase&fields=METRICS";
> | jq .
> {
> "metrics": [],
> "events": [],
> "createdtime": 1553695002014,
> "idprefix": 0,
> "type": "YARN_CONTAINER",
> "id": "container_e18_1553685341603_0006_01_000001",
> "info": {
> "UID":
> "ats!application_1553685341603_0006!YARN_CONTAINER!0!container_e18_1553685341603_0006_01_000001",
> "FROM_ID":
> "ats!hbase!QuasiMonteCarlo!1553695001394!application_1553685341603_0006!YARN_CONTAINER!0!container_e18_1553685341603_0006_01_000001"
> },
> "configs": {},
> "isrelatedto": {},
> "relatesto": {}
> }{code}
> NodeManager puts YARN_CONTAINER entities with CPU and MEMORY metrics but this
> is not shown in above output. Found NM container entities are set with
> entityIdPrefix as inverted container starttime whereas RM container entities
> are set with default 0. TimelineReader fetches only RM container entries.
> Confirmed with setting NM container entities entityIdPrefix to 0 same as RM
> (for testing purpose) and found metrics are shown.
> {code:java}
> "metrics": [
> {
> "type": "SINGLE_VALUE",
> "id": "MEMORY",
> "aggregationOp": "NOP",
> "values": {
> "1553774981355": 490430464
> }
> },
> {
> "type": "SINGLE_VALUE",
> "id": "CPU",
> "aggregationOp": "NOP",
> "values": {
> "1553774981355": 5
> }
> }
> ]{code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
