Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/12956 )
Change subject: IMPALA-8375: Add metrics for spill disk usage ...................................................................... IMPALA-8375: Add metrics for spill disk usage Added two new metrics tmp-file-mgr.scratch-space-bytes-used-high-water-mark & tmp-file-mgr.scratch-space-bytes-used for tracking HWM and current value for spilled bytes, respectively. A new class AtomicHighWaterMarkGauge was added to keep track of the HWM value. The new class also encapsulates a metric object which keeps track of the current value for the spilled bytes. The current value is incremented every time a new range is allocated from a temporary file. The current value for spilled bytes is decremented when a temporary file is closed. The new metrics are not updated when ranges are recycled from a file. We can add a new metric in future for keeping track of actual spilled bytes. The HWM value is updated whenever the current value is greater than the HWM value. Testing: - Added new unit tests to the metrics-test test case. - E2E testing for both the metrics by running concurrent spilling queries and ensuring that both the current value metric and the HWM metric were behaving as expected. Ran concurrent queries and monitored the metrics on the impala daemon's metric page. Change-Id: Ia1b3dd604c7234a8d8af34d70ca731544a46d298 Reviewed-on: http://gerrit.cloudera.org:8080/12956 Reviewed-by: Tim Armstrong <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- M be/CMakeLists.txt M be/src/runtime/tmp-file-mgr.cc M be/src/runtime/tmp-file-mgr.h M be/src/util/metrics-test.cc M be/src/util/metrics.h M common/thrift/metrics.json 6 files changed, 135 insertions(+), 3 deletions(-) Approvals: Tim Armstrong: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/12956 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ia1b3dd604c7234a8d8af34d70ca731544a46d298 Gerrit-Change-Number: 12956 Gerrit-PatchSet: 6 Gerrit-Owner: Abhishek Rawat <[email protected]> Gerrit-Reviewer: Abhishek Rawat <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Tim Armstrong <[email protected]>
