[
https://issues.apache.org/jira/browse/TEZ-3611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15899227#comment-15899227
]
Harish Jaiprakash commented on TEZ-3611:
----------------------------------------
Thanks @[~sseth].
* I'll make the change to return a list of event instead of a single event and
remove special handling of summary events. I made the changes to keep
HistoryEventTimelineConversion's API backwards compatible. Might not be a
requirement, please do confirm.
* The convertToTimelineEntity with false is being invoked by
ATSHistoryLoggingService. I can make the same changes there and keep both
loggers consisted w.r.t data format for ATS.
Yes, this puts the DAG_EXTRA_INFO into regular details file. The summary file
is used to get information for dag listing page where information in
DAG_EXTRA_INFO is not used. It helps by keeping the summary file small and
search and load would be faster on it.
Yes, this change will break the UI and we need to ensure that UI can work with
both formats of logging.
Yes, the entity type TEZ_DAG_ID is a summary entity type, so all events to this
entity Dag(submitted, intialized, started, finished, recovered)Events are
published into summary logs.
> Create lightweight summary events for ATS.
> ------------------------------------------
>
> Key: TEZ-3611
> URL: https://issues.apache.org/jira/browse/TEZ-3611
> Project: Apache Tez
> Issue Type: Bug
> Reporter: Harish Jaiprakash
> Assignee: Harish Jaiprakash
> Attachments: TEZ-3611.01.patch, TEZ-3611.02.patch, TEZ-3611.03.patch
>
>
> All DAG*Event are summary logs. DAGSubmittedEvent has dagPlan in it and
> DAGFinishedEvent has counters in it. And hence these events very large, which
> makes the dag landing page on the tez-ui very expensive.
> Split these events into two parts:
> * One which contains only data used by the tez-ui "All Dags" page which will
> go into summary log.
> * Another which will contain the current object, which will be part of the
> appropriate TimelineEntityGroupId.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
