[
https://issues.apache.org/jira/browse/YARN-6221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17381788#comment-17381788
]
Xiaomin Zhang commented on YARN-6221:
-------------------------------------
Greetings, [~gtcarrera9]. May I know if you are still working on this jira?
We observed this issue when running Isilon. For some reasons, getListing output
is in fixed order from HDFS, but it's not true for Isilon. For most cases,
domain log is not listed as the first from Isilon's getListing call. And when
this happens, domain log will be after other summary files to be parsed by ATS.
We have a patch to make sure the domain log be parsed before others, which is
supposed to fix this issue. Can you please help assign this Jira so I can
upload the patch? Thanks.
> Entities missing from ATS when summary log file info got returned to the ATS
> before the domain log
> --------------------------------------------------------------------------------------------------
>
> Key: YARN-6221
> URL: https://issues.apache.org/jira/browse/YARN-6221
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn
> Reporter: Sushmitha Sreenivasan
> Assignee: Li Lu
> Priority: Critical
>
> Events data missing for the following entities:
> curl -k --negotiate -u:
> http://<timeline_host>:8188/ws/v1/timeline/TEZ_APPLICATION_ATTEMPT/tez_appattempt_1487706062210_0012_000001
> {"events":[],"entitytype":"TEZ_APPLICATION_ATTEMPT","entity":"tez_appattempt_1487706062210_0012_000001","starttime":1487711606077,"domain":"Tez_ATS_application_1487706062210_0012","relatedentities":{"TEZ_DAG_ID":["dag_1487706062210_0012_2","dag_1487706062210_0012_1"]},"primaryfilters":{},"otherinfo":{}}
> {code:title=Timeline Server log entry}
> WARN timeline.TimelineDataManager
> (TimelineDataManager.java:doPostEntities(366)) - Skip the timeline entity: {
> id: tez_application_1487706062210_0012, type: TEZ_APPLICATION }
> org.apache.hadoop.yarn.exceptions.YarnException: Domain information of the
> timeline entity { id: tez_application_1487706062210_0012, type:
> TEZ_APPLICATION } doesn't exist.
> at
> org.apache.hadoop.yarn.server.timeline.security.TimelineACLsManager.checkAccess(TimelineACLsManager.java:122)
> at
> org.apache.hadoop.yarn.server.timeline.TimelineDataManager.doPostEntities(TimelineDataManager.java:356)
> at
> org.apache.hadoop.yarn.server.timeline.TimelineDataManager.postEntities(TimelineDataManager.java:316)
> at
> org.apache.hadoop.yarn.server.timeline.EntityLogInfo.doParse(LogInfo.java:204)
> at
> org.apache.hadoop.yarn.server.timeline.LogInfo.parsePath(LogInfo.java:156)
> at
> org.apache.hadoop.yarn.server.timeline.LogInfo.parseForStore(LogInfo.java:113)
> at
> org.apache.hadoop.yarn.server.timeline.EntityGroupFSTimelineStore$AppLogs.parseSummaryLogs(EntityGroupFSTimelineStore.java:682)
> at
> org.apache.hadoop.yarn.server.timeline.EntityGroupFSTimelineStore$AppLogs.parseSummaryLogs(EntityGroupFSTimelineStore.java:657)
> at
> org.apache.hadoop.yarn.server.timeline.EntityGroupFSTimelineStore$ActiveLogParser.run(EntityGroupFSTimelineStore.java:870)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
