Hello Kurt Deschler, Yida Wu, Michael Smith, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/23017
to look at the new patch set (#4).
Change subject: IMPALA-13945: Change hash trace to show each node's individual
contribution
......................................................................
IMPALA-13945: Change hash trace to show each node's individual contribution
Currently, the hash trace accumulates up the plan tree and is
displayed only for tuple cache nodes. This means that tuple cache
nodes high in a large plan can have hundreds of lines of hash trace
output without an indication of which contributions came from
which nodes.
This changes the hash trace in two ways:
1. It displays each plan node's individual contribution to the hash
trace. This only contains a summary of the hash contributed by
the child, so the hash trace does not accumulate up the plan tree.
Since each node is displaying its own contribution, the tuple
cache node does not display the hash trace itself.
2. This adds structure to the hash trace to include a comment for
each contribution to the hash trace. This allows a cleaner display
of the individual pieces of a node's hash trace. It also gives
extra information about the specific contributions into the hash.
It should be possible to trace the contribution through the plan
tree.
This also changes the output to only display the hash trace with
explain_level=EXTENDED or higher (i.e. it won't be displayed with
STANDARD).
Example output:
tuple cache hash trace:
TupleDescriptor 0: TTupleDescriptor(id:0, byteSize:0, numNullBytes:0,
tableId:1, tuplePath:[])
Table: TTableName(db_name:functional, table_name:alltypes)
PlanNode:
[TPlanNode(node_id:0, node_type:HDFS_SCAN_NODE, num_children:0,
limit:-1, row_tuples:[0], nullable_tu]
[ples:[false], disable_codegen:false, pipelines:[],
hdfs_scan_node:THdfsScanNode(tuple_id:0, random_r]
[eplica:false, use_mt_scan_node:false, is_partition_key_scan:false,
file_formats:[]), resource_profil]
[e:TBackendResourceProfile(min_reservation:0, max_reservation:0))]
Query options hash: TQueryOptionsHash(hi:-2415313890045961504,
lo:-1462668909363814466)
Testing:
- Modified TupleCacheInfoTest and TupleCacheTest to use the new hash trace
Change-Id: If53eda24e7eba264bc2d2f212b63eab9dc97a74c
---
M fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java
M fe/src/main/java/org/apache/impala/planner/PlanNode.java
M fe/src/main/java/org/apache/impala/planner/TupleCacheInfo.java
M fe/src/main/java/org/apache/impala/planner/TupleCacheNode.java
M fe/src/test/java/org/apache/impala/planner/TupleCacheInfoTest.java
M fe/src/test/java/org/apache/impala/planner/TupleCacheTest.java
M testdata/workloads/functional-query/queries/QueryTest/explain-level1.test
7 files changed, 190 insertions(+), 90 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/17/23017/4
--
To view, visit http://gerrit.cloudera.org:8080/23017
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: If53eda24e7eba264bc2d2f212b63eab9dc97a74c
Gerrit-Change-Number: 23017
Gerrit-PatchSet: 4
Gerrit-Owner: Joe McDonnell <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: Joe McDonnell <[email protected]>
Gerrit-Reviewer: Kurt Deschler <[email protected]>
Gerrit-Reviewer: Michael Smith <[email protected]>
Gerrit-Reviewer: Yida Wu <[email protected]>
