[
https://issues.apache.org/jira/browse/TEZ-1652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14343314#comment-14343314
]
Jonathan Eagles commented on TEZ-1652:
--------------------------------------
[~Sreenath], totally agree with you about the UI design doc. We definitely need
to be aiming at something while changing the look and functionality of UI. That
should be the next step before any UI look and functionality changes are made.
In this case, I am aiming at fixing the bug filed both here and recently by
Rohini in TEZ-2153. Essentially there are two major changes needed. In that
case, we aren't showing all events and the performance rendering for large jobs
was very bad.
*Addressing the bugs*
# We need all task attempts available. This was done by moving the load from
the router to the controller/component, reusing the design and approach from
graph-view's underlying paginated content.
# There is a huge performance bottleneck in the
TaskAttemptSerializer:normalizePayload
{code}
[].push.apply(normalizedPayload.counterGroups, n.counterGroups);
[].push.apply(normalizedPayload.counters, n.counters);
{code}
These two lines have been removed from the
SwimlaneTaskAttemptSerializer:normalizePayload.
*Performance Impact*
With a 3000 task attempts the dag swimlane renders in 6.59 seconds with
swimlane_task_attempt and 37.08 seconds with task_attempt. Best is if we can
reap that render performance across the whole project. However, the
normalization is providing some value to other views, so I am hesitant to make
the change globally.
Let me know, if I have sufficiently addressed your concerns over this bug and
future direction of the Swimlanes view.
> Large job support for Swimlane view
> -----------------------------------
>
> Key: TEZ-1652
> URL: https://issues.apache.org/jira/browse/TEZ-1652
> Project: Apache Tez
> Issue Type: Sub-task
> Reporter: Jonathan Eagles
> Assignee: Jonathan Eagles
> Attachments: TEZ-1652-v1.patch, TEZ-1652.1.patch, TEZ-1652.2.patch
>
>
> Issue is that by default, Timeline Server only returns 100 max entity results
> per query. For query for all task attempts for dag. A limit should be high
> enough to get all entries and yet provide responsiveness.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
