[
https://issues.apache.org/jira/browse/IMPALA-14958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18079343#comment-18079343
]
ASF subversion and git services commented on IMPALA-14958:
----------------------------------------------------------
Commit f7b4601482ab698ff524a68e89c9485299879062 in impala's branch
refs/heads/master from stiga-huang
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=f7b460148 ]
IMPALA-14958: Fix missing lifecycle events for some nodes
Node Lifecycle Events in profile are incomplete for KuduScanNodeMt,
IcebergMetadataScanNode and SystemTableScanNode. It's fixed by adding
the invocation of ScopedOpenEventAdder and ScopedGetNextEventAdder in
the corresponding Open and GetNext implementations.
Also adds/adjusts the total time counter in some nodes.
Testing
- Added e2e tests
- Improved the timeline checking by verifying all the occurrences.
Assisted-by: Composer 2
Change-Id: I39abc3a74df7b4ea3383f104fe1a74ff09f18324
Reviewed-on: http://gerrit.cloudera.org:8080/24281
Reviewed-by: Michael Smith <[email protected]>
Reviewed-by: Joe McDonnell <[email protected]>
Tested-by: Quanlong Huang <[email protected]>
> Missing Node Lifecycle Events in some ExecNodes
> -----------------------------------------------
>
> Key: IMPALA-14958
> URL: https://issues.apache.org/jira/browse/IMPALA-14958
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Reporter: Quanlong Huang
> Assignee: Quanlong Huang
> Priority: Critical
>
> Some ExecNode implementations don't invoke
> [ScopedOpenEventAdder|https://github.com/apache/impala/blob/a44f72d8f80677509a98439a5b46fa2cbb933db7/be/src/exec/exec-node-util.h#L31]
> and
> [ScopedGetNextEventAdder|https://github.com/apache/impala/blob/a44f72d8f80677509a98439a5b46fa2cbb933db7/be/src/exec/exec-node-util.h#L50],
> which leads to weird timelines in the profile that just have the "Closed"
> event. KuduScanNodeMt is an example.
> KuduScanNodeMt (MT_DOP>0)
> {noformat}
> KUDU_SCAN_NODE (id=0):
> Table Name: functional_kudu.alltypes
> Node Lifecycle Event Timeline: 13.588ms
> - Closed: 13.588ms (13.588ms){noformat}
> KuduScanNode (MT_DOP=0) is fine.
> {noformat}
> KUDU_SCAN_NODE (id=0):
> Table Name: functional_kudu.alltypes
> Node Lifecycle Event Timeline: 26.370ms
> - Open Started: 15.425ms (15.425ms)
> - Open Finished: 22.982ms (7.557ms)
> - First Batch Requested: 22.987ms (5.073us)
> - First Batch Returned: 26.349ms (3.361ms)
> - Last Batch Returned: 26.351ms (1.838us)
> - Closed: 26.370ms (18.892us){noformat}
> IcebergMetadataScanNode and SystemTableScanNode also have this issue:
> {noformat}
> Sql Statement: select * from
> functional_parquet.iceberg_query_metadata.entries
> ICEBERG_METADATA_SCAN_NODE (id=0):
> Node Lifecycle Event Timeline: 213.956ms
> - Closed: 213.956ms (213.956ms)
> Sql Statement: select * from sys.impala_query_live
> SYSTEM_TABLE_SCAN_NODE (id=0):
> Node Lifecycle Event Timeline: 16.510ms
> - Closed: 16.510ms (16.510ms){noformat}
> This is misleading for HBO to check whether the node actually completes.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
