Hello Andrew Sherman, Lars Volker, David Rorke, Balazs Jeszenszky, Jiawei Wang,
Tim Armstrong, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/14776
to look at the new patch set (#10).
Change subject: IMPALA-7550: Add documentation to profile counters
......................................................................
IMPALA-7550: Add documentation to profile counters
This change changes the way developers define profile counters by
generating counters from a counter registry. All the profile counters
will be register there first and then used in the same way as before.
By doing so, we will be able to manage profile counters in a way that
we can define profile counters documentation. For example:
Declaration:
PROFILE_DEFINE_COUNTER(NumBackends, STABLE_HIGH, TUnit::UNIT,
"Number of backends running this query.");
Initialization:
COUNTER_SET(PROFILE_NumBackends.Instantiate(query_profile_), num_backends);
This shall be how we define a NumBackends counter. It follows with
its significance, type, description in the declaration part.
Users now will be able to view profile counters documentation under
query_profile page, there is a Profile Documentation button which
leads to /profile_docs.
More details:
This commit did the following refactors on profile counters.
1. Add a singleton registry for runtime profile counters prototypes,
similiar to what Kudu does for metrics. This allows us to generate
profile documentation for all counters from the code. We add
/profile_docs and a correspoding UI for the documentation of profile
counters.
2. Profile counters are also annotated with their significance to users.
* STABLE_HIGH - High level and stable counters, always useful for measuring
query performance and status. Counters that everyone is interested. should
rarely change and if it does we will make some effort to notify users.
* STABLE_LOW - Low level and stable counters. Interesting counters to monitor
and analyze by machine. It will probably be interesting under some
circumstances for users.
* Unstable - Unstable but useful. Useful to understand query performance,
but subject to change, particularly if the implementation changes.
E.g. MaterializeTupleTimer
* Debug - Debugging counters. Generally not useful to users of Impala,
the main use case is low-level debugging. Can be hidden to reduce noise
for most consumers of profiles.
3. We have around 250 counters. This commit did the replacement in
scan-node and hdfs-scan-node-base and coordinator.
Concers:
The downside is that we will have duplicate comments of query profiles
both in the header file and the .cc file.
Additionally a (arguably good) limitation is that profile counter names
need to be unique.
Change-Id: Idc03faddb27754001290bb6d899840e2cbe7ccb7
---
M be/src/exec/hbase-scan-node.cc
M be/src/exec/hbase-scan-node.h
M be/src/exec/hdfs-scan-node-base.cc
M be/src/exec/hdfs-scan-node-base.h
M be/src/exec/kudu-scan-node-base.cc
M be/src/exec/scan-node.cc
M be/src/exec/scan-node.h
M be/src/runtime/coordinator-backend-state.cc
M be/src/runtime/coordinator.cc
D be/src/util/debug-counters.h
M be/src/util/default-path-handlers.cc
M be/src/util/runtime-profile-counters.h
M be/src/util/runtime-profile.cc
A www/profile_docs.tmpl
M www/query_profile.tmpl
15 files changed, 678 insertions(+), 273 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/76/14776/10
--
To view, visit http://gerrit.cloudera.org:8080/14776
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Idc03faddb27754001290bb6d899840e2cbe7ccb7
Gerrit-Change-Number: 14776
Gerrit-PatchSet: 10
Gerrit-Owner: Jiawei Wang <[email protected]>
Gerrit-Reviewer: Andrew Sherman <[email protected]>
Gerrit-Reviewer: Balazs Jeszenszky <[email protected]>
Gerrit-Reviewer: David Rorke <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: Jiawei Wang <[email protected]>
Gerrit-Reviewer: Jiawei Wang <[email protected]>
Gerrit-Reviewer: Lars Volker <[email protected]>
Gerrit-Reviewer: Tim Armstrong <[email protected]>
