Hello Quanlong Huang, Michael Smith, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/23794
to look at the new patch set (#2).
Change subject: IMPALA-6556: Show in-flight table metadata loading on Catalog
WebUI
......................................................................
IMPALA-6556: Show in-flight table metadata loading on Catalog WebUI
This patch extends the /operations page in catalogd WebUI to show
in-flight table metadata loading operations with detailed timeline
events and real-time progress tracking. This helps users understand
if a query is hung or just waiting for metadata to load. Covers all
table loading operations, including REFRESH, query-triggered metadata
loading, background/async loading, and INVALIDATE METADATA.
The following information is displayed for each loading operation:
- Table name: Fully qualified table name
- Query ID: The query that triggered the metadata loading
- Submit time: When the task was put into the table loading queue
- Start time: When Table.load() is actually invoked
- Waiting time: Duration between submit time and start time
- Duration: Time elapsed since start
- Progress: Real-time partition counts
- Timeline: Entire timeline details in modal
Testing:
- Adds tests to verify table loading structure, field presence, and value
formats
- Validates in-flight progress tracking and timeline formatting with multiple
events
- Manually tested on tables with ~3000 partitions
Change-Id: I8486b0deabd1bb34e3d83e11bd3350aaa7d696f4
---
M be/src/catalog/catalog-server.cc
M common/thrift/JniCatalog.thrift
M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java
M fe/src/main/java/org/apache/impala/catalog/ParallelFileMetadataLoader.java
M fe/src/main/java/org/apache/impala/catalog/TableLoader.java
M fe/src/main/java/org/apache/impala/catalog/TableLoadingMgr.java
M
fe/src/main/java/org/apache/impala/catalog/monitor/CatalogOperationTracker.java
A fe/src/main/java/org/apache/impala/catalog/monitor/TableLoadingTracker.java
M fe/src/main/java/org/apache/impala/util/MetaStoreUtil.java
M tests/webserver/test_web_pages.py
M www/catalog_operations.tmpl
11 files changed, 1,281 insertions(+), 4 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/94/23794/2
--
To view, visit http://gerrit.cloudera.org:8080/23794
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I8486b0deabd1bb34e3d83e11bd3350aaa7d696f4
Gerrit-Change-Number: 23794
Gerrit-PatchSet: 2
Gerrit-Owner: Arnab Karmakar <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: Michael Smith <[email protected]>
Gerrit-Reviewer: Quanlong Huang <[email protected]>
