Abhishek Rawat has uploaded a new patch set (#5). ( http://gerrit.cloudera.org:8080/12956 )
Change subject: IMPALA-8375: Add metrics for spill disk usage ...................................................................... IMPALA-8375: Add metrics for spill disk usage Added two new metrics tmp-file-mgr.scratch-space-bytes-used-high-water-mark & tmp-file-mgr.scratch-space-bytes-used for tracking HWM and current value for spilled bytes, respectively. A new class AtomicHighWaterMarkGauge was added to keep track of the HWM value. The new class also encapsulates a metric object which keeps track of the current value for the spilled bytes. The current value is incremented every time a new range is allocated from a temporary file. The current value for spilled bytes is decremented when a temporary file is closed. The new metrics are not updated when ranges are recycled from a file. We can add a new metric in future for keeping track of actual spilled bytes. The HWM value is updated whenever the current value is greater than the HWM value. Testing: - Added new unit tests to the metrics-test test case. - E2E testing for both the metrics by running concurrent spilling queries and ensuring that both the current value metric and the HWM metric were behaving as expected. Ran concurrent queries and monitored the metrics on the impala daemon's metric page. Change-Id: Ia1b3dd604c7234a8d8af34d70ca731544a46d298 --- M be/CMakeLists.txt M be/src/runtime/tmp-file-mgr.cc M be/src/runtime/tmp-file-mgr.h M be/src/util/metrics-test.cc M be/src/util/metrics.h M common/thrift/metrics.json 6 files changed, 135 insertions(+), 3 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/56/12956/5 -- To view, visit http://gerrit.cloudera.org:8080/12956 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ia1b3dd604c7234a8d8af34d70ca731544a46d298 Gerrit-Change-Number: 12956 Gerrit-PatchSet: 5 Gerrit-Owner: Abhishek Rawat <ara...@cloudera.com> Gerrit-Reviewer: Abhishek Rawat <ara...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Tim Armstrong <tarmstr...@cloudera.com>
