Hello Quanlong Huang, Yongzhi Chen, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/14600
to look at the new patch set (#6).
Change subject: IMPALA-9109: Add top-k metadata loading ranking on catalogd UI
......................................................................
IMPALA-9109: Add top-k metadata loading ranking on catalogd UI
Add functions in CatalogUsageMonitor to monitor and report the
catalog usage of the tables have the longest metadata loading
time(Including maximum, median, 75th-ile, 95th-ile, 99th-ile time).
Set default tables loading metrics capacity to 100.
However, there might be a problem here because we only keep the
capacity size to 100. For example, there might be case like a
table has higher median loading time but has lower Maximum
loading time which cannot make itself to the Top-100. For now,
we will ignore case like that because we are aiming to find
the tables with maximum longest loading time.
Add the sorted table in Catalog server web-ui. The loading
time is sorted by the maximum from load_duration metrics. But
users can sort by other metrics in catalogd debug UI.
Testing:
- Add end-to-end test for webpage to verify the label and text
exist in catalog debug page. Verify all fields are in JSON response
- Launch Impala and activate some tables to see the table loading
time shown successfully on the catalog debug UI page.
Change-Id: I9305a867d7053cde9acc42dae6e47ee440f1a8bf
---
M be/src/catalog/catalog-server.cc
M be/src/catalog/catalog-server.h
M common/thrift/JniCatalog.thrift
M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java
M fe/src/main/java/org/apache/impala/catalog/CatalogUsageMonitor.java
M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java
M fe/src/main/java/org/apache/impala/catalog/Table.java
M tests/webserver/test_web_pages.py
M www/catalog.tmpl
M www/scripts/util.js
10 files changed, 269 insertions(+), 13 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/00/14600/6
--
To view, visit http://gerrit.cloudera.org:8080/14600
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I9305a867d7053cde9acc42dae6e47ee440f1a8bf
Gerrit-Change-Number: 14600
Gerrit-PatchSet: 6
Gerrit-Owner: Jiawei Wang <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: Jiawei Wang <[email protected]>
Gerrit-Reviewer: Quanlong Huang <[email protected]>
Gerrit-Reviewer: Yongzhi Chen <[email protected]>
