Hello Philip Zeyliger, Csaba Ringhofer, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/12229
to look at the new patch set (#3).
Change subject: IMPALA-7731: Add Read/Exchange counters to profile
......................................................................
IMPALA-7731: Add Read/Exchange counters to profile
Selective scans (and by extension selective fragment instances)
take higher performance hits when reading data remotely. They can
be identified by a low ratio between data being transmitted vs data
being read from HDFS.
This change adds several counters to the profile to make it easier to
identify queries based on their scan instance selectivity.
* TotalBytesRead - The total number of bytes read by a query.
* TotalBytesSent - The total number of bytes sent by a query in
exchange nodes. Does not include remote reads, data written to disk,
or data sent to the client.
* TotalScanBytesSent - The total number of bytes sent by fragment
instances that had a scan node in their plan.
* TotalInnerBytesSent - The total number of bytes sent by fragment
instances that did not have a scan node in their plan, i.e. that
received their input data from other instances through exchange node.
* ExchangeScanRatio - The ratio between TotalScanBytesSent and
TotalBytesRead, i.e. the selectivity over all fragment instances that
had a scan node in their plan. This counter is also added to each
fragment instance.
* InnerNodeSelectivityRatio - The ratio between bytes sent by instances
with a scan node in their plan and instances without a scan node in
their plan. This indicates how well the inner nodes of the execution
plan reduced the data volume.
Change-Id: Ife7ec78fe42558429c1cbe6e5eba79842bffd648
---
M be/src/exec/scan-node.cc
M be/src/runtime/coordinator-backend-state.cc
M be/src/runtime/coordinator-backend-state.h
M be/src/runtime/coordinator.cc
M be/src/runtime/coordinator.h
M be/src/runtime/fragment-instance-state.cc
M be/src/runtime/krpc-data-stream-sender.cc
M be/src/runtime/runtime-state.cc
M be/src/runtime/runtime-state.h
M be/src/util/pretty-printer.h
M tests/query_test/test_observability.py
11 files changed, 112 insertions(+), 2 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/29/12229/3
--
To view, visit http://gerrit.cloudera.org:8080/12229
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ife7ec78fe42558429c1cbe6e5eba79842bffd648
Gerrit-Change-Number: 12229
Gerrit-PatchSet: 3
Gerrit-Owner: Lars Volker <l...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <csringho...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
Gerrit-Reviewer: Lars Volker <l...@cloudera.com>
Gerrit-Reviewer: Philip Zeyliger <phi...@cloudera.com>
