Bikramjeet Vig has uploaded this change for review. (
http://gerrit.cloudera.org:8080/15663
Change subject: IMPALA-9422: Re-visit and improve join node and builder's
counters
......................................................................
IMPALA-9422: Re-visit and improve join node and builder's counters
This patch makes the following changes:
- All code executed inside the builder that gets executed exactly once
during query execution is attributed to the builder. This also include
public calls to the builder that are used as synchronization points
for shared builds. The serial execution phase in these methods are
always executed once regardless of the builder execution mode (namely
single-threaded, parallel execution, separate build sink).
- Also makes sure there is no double counting of total time in builder.
- BuildTime counter has been removed from the join node's profile in
favor of the builder's total time.
- BuildRowsPartitioned from the builder is equivalent to BuildRows in
the join node and hence that counter has been moved to the builders.
An example of the new profiles:
HASH_JOIN_NODE (id=2):(Total: 29s994ms, non-child: 0.000ns, % non-child: 0.00%)
- PeakMemoryUsage: 512.09 MB (536969472)
- ProbeRows: 2.00M (2000405)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 348.899ms
- RowsReturned: 2.00M (2000405)
- RowsReturnedRate: 66.69 K/sec
Buffer pool:
- AllocTime: 3.290ms
- CompressionTime: 0.000ns
- CumulativeAllocationBytes: 512.00 MB (536870912)
- CumulativeAllocations: 240 (240)
- EncryptionTime: 0.000ns
- PeakReservation: 512.00 MB (536870912)
- PeakUnpinnedBytes: 0
- PeakUsedReservation: 512.00 MB (536870912)
- ReadIoBytes: 0
- ReadIoOps: 0 (0)
- ReadIoWaitTime: 0.000ns
- SystemAllocTime: 2.955ms
- WriteIoBytes: 0
- WriteIoOps: 0 (0)
- WriteIoWaitTime: 0.000ns
Hash Join Builder (join_node_id=2):(Total: 1s873ms, non-child: 0.000ns, %
non-child: 0.00%)
- BuildRows: 2.00M (2000405)
- BuildRowsPartitionTime: 1s222ms
- HashTablesBuildTime: 647.959ms
- LargestPartitionPercent: 6 (6)
- MaxPartitionLevel: 0 (0)
- NumHashTableBuildsSkipped: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 44.12 KB (45184)
- RepartitionTime: 0.000ns
- SpilledPartitions: 0 (0)
Hash Table:
- HashBuckets: 4.19M (4194304)
- HashCollisions: 898 (898)
- Probes: 4.00M (4000810)
- Resizes: 0 (0)
- Travel: 1.61M (1607615)
NESTED_LOOP_JOIN_NODE (id=3):(Total: 2s042ms, non-child: 0.000ns, % non-child:
0.00%)
ExecOption: Join Build-Side Prepared Asynchronously
Node Lifecycle Event Timeline: 2s326ms
- Open Started: 266.030ms (266.030ms)
- Open Finished: 2s160ms (1s894ms)
- First Batch Requested: 2s160ms (4.652us)
- First Batch Returned: 2s161ms (513.662us)
- Last Batch Returned: 2s324ms (163.194ms)
- Closed: 2s326ms (2.685ms)
- PeakMemoryUsage: 51.75 MB (54263808)
- ProbeRows: 3 (3)
- ProbeTime: 0.000ns
- RowsReturned: 301.15K (301146)
- RowsReturnedRate: 147.45 K/sec
Nested Loop Join Builder:(Total: 486.669ms, non-child: 0.000ns, % non-child:
0.00%)
- BuildRows: 100.38K (100382)
- PeakMemoryUsage: 51.72 MB (54235136)
Change-Id: I604075a2c8efcff26705fb39672f29f309b2ed97
---
M be/src/exec/blocking-join-node.cc
M be/src/exec/blocking-join-node.h
M be/src/exec/join-builder.h
M be/src/exec/nested-loop-join-builder.cc
M be/src/exec/partitioned-hash-join-builder.cc
M be/src/exec/partitioned-hash-join-builder.h
M be/src/exec/partitioned-hash-join-node.cc
7 files changed, 99 insertions(+), 83 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/63/15663/1
--
To view, visit http://gerrit.cloudera.org:8080/15663
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I604075a2c8efcff26705fb39672f29f309b2ed97
Gerrit-Change-Number: 15663
Gerrit-PatchSet: 1
Gerrit-Owner: Bikramjeet Vig <[email protected]>
