[
https://issues.apache.org/jira/browse/TEZ-3611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15898339#comment-15898339
]
Siddharth Seth commented on TEZ-3611:
-------------------------------------
Minor comments on the patch.
- Should we have HistoryEventTimelineConversion return a list from
convertToTimelineEntity - instead of exposing summary vs non-summary events in
this class?
- Nothing invokes convertToTimelineEntity with a value of false? (That was for
backward compatibility of the UI I believe?)
Question: This puts DAG_EXTRA_INFO into the regular details file, correct? Does
it make sense to put DAG_EXTRA_INFO into the summary as well, (except not part
of DAG_ID - This is really just to get past the inability to retrieve specific
fields from ATS).
Other than this, changes look good to me.
Think we need to wait for the UI part of the change to be in place before
committing this?
Unrelated question: I didn't realize there's additional information like Vertex
list (without task count) logged as part of TEZ_DAG_ID. Assuming all the events
- recovery, dag initialized, etc - all pubish to ATS summry, since they all
update TEZ_DAG_ID?
> Create lightweight summary events for ATS.
> ------------------------------------------
>
> Key: TEZ-3611
> URL: https://issues.apache.org/jira/browse/TEZ-3611
> Project: Apache Tez
> Issue Type: Bug
> Reporter: Harish Jaiprakash
> Assignee: Harish Jaiprakash
> Attachments: TEZ-3611.01.patch, TEZ-3611.02.patch, TEZ-3611.03.patch
>
>
> All DAG*Event are summary logs. DAGSubmittedEvent has dagPlan in it and
> DAGFinishedEvent has counters in it. And hence these events very large, which
> makes the dag landing page on the tez-ui very expensive.
> Split these events into two parts:
> * One which contains only data used by the tez-ui "All Dags" page which will
> go into summary log.
> * Another which will contain the current object, which will be part of the
> appropriate TimelineEntityGroupId.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
