Hello Philip Zeyliger, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/12229
to look at the new patch set (#2).
Change subject: IMPALA-7731: Add TxRatio counter to instances and query profile
......................................................................
IMPALA-7731: Add TxRatio counter to instances and query profile
Selective scans (and by extension selective fragment instances)
take higher performance hits when reading data remotely. They can
be identified by a low ratio between data being transmitted vs data
being read from HDFS.
This change adds the TxRatio counter to profiles for fragment instances
and the overall query. It is computed as the ratio of data transmitted
by instances with a scan over the data read by all scans of the
instance.
For example, if the scan node(s) of a fragment instance read 100MB from
disk and the instance (after decompression) sends 300MB over the network
through an exchange, that instance would have a TxRatio of 3.
For the whole query, we can compute the TxRatio in the same way by
aggregating the bytes read and sent for instances that contain a scan
node.
This change adds TxRatio computation and counters to both the fragments
instances on the executor side, and the overall query on the coordinator
side. Both take a similar approach in the RuntimeState and Coordinator
respectively.
Change-Id: Ife7ec78fe42558429c1cbe6e5eba79842bffd648
---
M be/src/exec/scan-node.cc
M be/src/runtime/coordinator-backend-state.cc
M be/src/runtime/coordinator-backend-state.h
M be/src/runtime/coordinator.cc
M be/src/runtime/coordinator.h
M be/src/runtime/fragment-instance-state.cc
M be/src/runtime/krpc-data-stream-sender.cc
M be/src/runtime/runtime-state.cc
M be/src/runtime/runtime-state.h
M be/src/util/pretty-printer.h
M tests/query_test/test_observability.py
11 files changed, 97 insertions(+), 2 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/29/12229/2
--
To view, visit http://gerrit.cloudera.org:8080/12229
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ife7ec78fe42558429c1cbe6e5eba79842bffd648
Gerrit-Change-Number: 12229
Gerrit-PatchSet: 2
Gerrit-Owner: Lars Volker <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: Lars Volker <[email protected]>
Gerrit-Reviewer: Philip Zeyliger <[email protected]>
