[
https://issues.apache.org/jira/browse/YARN-8625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16839177#comment-16839177
]
Prabhu Joseph commented on YARN-8625:
-------------------------------------
[~eepayne] Suspect Resourcemanager address and port is used. Can you check with
ApplicationHistoryServer (TimelineServer) address and port 8188.
REST Api - http://<ApplicationHistoryServer
IP>:8188/ws/v1/applicationhistory/apps
UI - http://<ApplicationHistoryServer IP>:8188/applicationhistory
Some Troubleshooting steps and attached screenshots for reference.
{code}
1. Check if TimelineServer 1.5 is Running and using port 8188
[yarn-ats@yarn-ats-1 hdfs]$ yarn-daemon.sh start timelineserver
[yarn-ats@yarn-ats-1 hdfs]$ ps -ef | grep ApplicationHistoryServer
yarn-ats 23243 1 24 07:27 pts/0 00:00:13
/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.212.b04-0.el7_6.x86_64/jre//bin/java
-Dproc_timelineserver -Djava.net.preferIPv4Stack=true
-Dyarn.log.dir=/HADOOP/hadoop-3.2.0/logs
-Dyarn.log.file=hadoop-yarn-ats-timelineserver-yarn-ats-1.log
-Dyarn.home.dir=/HADOOP/hadoop-3.2.0 -Dyarn.root.logger=INFO,console
-Djava.library.path=/HADOOP/hadoop-3.2.0/lib/native
-Dhadoop.log.dir=/HADOOP/hadoop-3.2.0/logs
-Dhadoop.log.file=hadoop-yarn-ats-timelineserver-yarn-ats-1.log
-Dhadoop.home.dir=/HADOOP/hadoop-3.2.0 -Dhadoop.id.str=yarn-ats
-Dhadoop.root.logger=INFO,RFA -Dhadoop.policy.file=hadoop-policy.xml
-Dhadoop.security.logger=INFO,NullAppender
org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryServer
[yarn-ats@yarn-ats-1 hdfs]$ netstat -plant | grep 8188
tcp 0 0 0.0.0.0:8188 0.0.0.0:* LISTEN
23243/java
2. Run a sample MR job
3. Test http request with curl using ApplicationHistoryServer and port 8188
[ambari-qa@yarn-ats-3 ~]$ curl
http://yarn-ats-1:8188/ws/v1/applicationhistory/apps | jq .
{
"app": [
{
"appId": "application_1557818827301_0001",
"currentAppAttemptId": "appattempt_1557818827301_0001_000001",
.........
"aggregateResourceAllocation": "67374 MB-seconds, 34 vcore-seconds",
"aggregatePreemptedResourceAllocation": "0 MB-seconds, 0 vcore-seconds"
}
{code}
> Aggregate Resource Allocation for each job is not present in ATS
> ----------------------------------------------------------------
>
> Key: YARN-8625
> URL: https://issues.apache.org/jira/browse/YARN-8625
> Project: Hadoop YARN
> Issue Type: Bug
> Components: ATSv2
> Affects Versions: 2.7.4
> Reporter: Prabhu Joseph
> Assignee: Prabhu Joseph
> Priority: Major
> Attachments: 0001-YARN-8625.patch, 0002-YARN-8625.patch, yarn-site.xml
>
>
> Aggregate Resource Allocation shown on RM UI for finished job is very useful
> metric to understand how much resource a job has consumed. But this does not
> get stored in ATS.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
