Lars Volker has uploaded this change for review. ( http://gerrit.cloudera.org:8080/12229
Change subject: IMPALA-7731: Add TxRatio counter to instances and query profile ...................................................................... IMPALA-7731: Add TxRatio counter to instances and query profile Selective scans (and by extension selective fragment instances) take higher performance hits when reading data remotely. They can be identified by a low ratio between data being transmitted vs data being read from HDFS. This change adds the TxRatio counter to profiles for fragment instances and the overall query. It is computed as the ratio of data transmitted by instances with a scan over the data read by all scans of the instance. For example, if the scan node(s) of a fragment instance read 100MB from disk and the instance (after decompression) sends 300MB over the network through an exchange, that instance would have a TxRatio of 3. For the whole query, we can compute the TxRatio in the same way by aggregating the bytes read and sent for instances that contain a scan node. This change adds TxRatio computation and counters to both the fragments instances on the executor side, and the overall query on the coordinator side. Both take a similar approach in the RuntimeState and Coordinator respectively. Change-Id: Ife7ec78fe42558429c1cbe6e5eba79842bffd648 --- M be/src/exec/scan-node.cc M be/src/runtime/coordinator-backend-state.cc M be/src/runtime/coordinator-backend-state.h M be/src/runtime/coordinator.cc M be/src/runtime/coordinator.h M be/src/runtime/fragment-instance-state.cc M be/src/runtime/krpc-data-stream-sender.cc M be/src/runtime/runtime-state.cc M be/src/runtime/runtime-state.h M be/src/util/pretty-printer.h M tests/query_test/test_observability.py 11 files changed, 81 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/29/12229/1 -- To view, visit http://gerrit.cloudera.org:8080/12229 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ife7ec78fe42558429c1cbe6e5eba79842bffd648 Gerrit-Change-Number: 12229 Gerrit-PatchSet: 1 Gerrit-Owner: Lars Volker <[email protected]>
