[
https://issues.apache.org/jira/browse/IMPALA-13624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17906660#comment-17906660
]
Surya Hebbar commented on IMPALA-13624:
---------------------------------------
The current list representation is verbose, it contains the timestamps for all
instances and all events without aggregation in the following manner.
{code}
AGGREGATION_NODE (id=47) [12 instances]:(Total: 1s728ms, non-child: 2.018ms, %
non-child: 0.12%)
Node Lifecycle Event Timeline[0]: 6s149ms
- Open Started: 4s093ms (4s093ms)
- Open Finished: 6s149ms (2s055ms)
- First Batch Requested: 6s149ms (12.889us)
- First Batch Returned: 6s149ms (2.147us)
- Last Batch Returned: 6s149ms (1.221us)
- Closed: 6s149ms (32.706us)
Node Lifecycle Event Timeline[1]: 6s149ms
- Open Started: 4s093ms (4s093ms)
- Open Finished: 6s149ms (2s056ms)
- First Batch Requested: 6s149ms (13.074us)
- First Batch Returned: 6s149ms (2.056us)
- Last Batch Returned: 6s149ms (148.000ns)
- Closed: 6s149ms (35.318us)
Node Lifecycle Event Timeline[2]: 6s149ms
- Open Started: 4s093ms (4s093ms)
- Open Finished: 6s149ms (2s055ms)
- First Batch Requested: 6s149ms (13.658us)
- First Batch Returned: 6s149ms (4.752us)
- Last Batch Returned: 6s149ms (118.000ns)
- Closed: 6s149ms (27.656us)
Node Lifecycle Event Timeline[3]:
......
......
{code}
With this format, it becomes difficult to analyze a particular phase across
instances.
Hence, a format that enhances aggregation of timestamps across instances
provides better insights into overall performance.
The suggested aggregate representation is the following.
{code:java}
AGGREGATION_NODE (id=47) [13 instances] : (Total: 2s525ms, non-child: 0.000ns,
% non-child: 0.00%)
Node Lifecycle Event Timeline Summary : 4s094ms
- Open Started: 2s850ms (2850ms)
Div. Min Max Avg # Inst. Count
20% 2000ms 2133ms 2050ms 3
40% 2200ms 2300ms 2250ms 4
60% 2400ms 2500ms 2450ms 2
80% 2700ms 2800ms 2750ms 2
100% 2850ms 2900ms 2875ms 2
- Open Finished: 4s094ms (2s269ms)
Div. Min Max Avg # Inst. Count
20% 3000ms 3133ms 3050ms 3
......
- First Batch Requested: 4s094ms (10.517us)
......
- First Batch Returned: 4s094ms (1.695us)
......
- Last Batch Returned: 4s094ms (104.000ns)
......
- Closed: 4s094ms (20.656us)
......
{code}
> Implement textual representation of aggregate event sequences
> -------------------------------------------------------------
>
> Key: IMPALA-13624
> URL: https://issues.apache.org/jira/browse/IMPALA-13624
> Project: IMPALA
> Issue Type: New Feature
> Reporter: Surya Hebbar
> Assignee: Surya Hebbar
> Priority: Major
> Attachments: list_representation.txt
>
>
> Currently, the node lifecycle event timeline among fragment instances is
> expressed in list format, without any form of summarization.
> Now, with the inclusion of aggregate metrics in the experimental profile in
> JSON format, it is possible to display them in a comprehensive format, which
> would help in skew analysis or other such cases.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
