[
https://issues.apache.org/jira/browse/IMPALA-9118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17779702#comment-17779702
]
ASF subversion and git services commented on IMPALA-9118:
---------------------------------------------------------
Commit b6bdf4c525acfe8b35928d43155cf3bea9be26cb in impala's branch
refs/heads/master from stiga-huang
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=b6bdf4c52 ]
IMPALA-9118: Show catalog operation details in catalogd webUI
This patch extends the /operations page in catalogd WebUI to show the
in-flight and finished catalog operations. The following fields are
shown for each operation:
- Thread ID
- Query ID
- Client IP
- Coordinator
- Type
- Target
- User
- Start Time
- End Time (only shown for finished operations)
- Duration
- Status
- Details
Finished operation records are currently kept in memory and limited by
the catalog_operation_log_size flag (defaults to 100).
To collect the above fields, this patch extends
TCatalogServiceRequestHeader to contain the coordinator hostname. Also
fix some catalog RPCs that didn't fill these fields.
Tests:
- Add e2e test in custom_cluster/test_web_pages.py
- Manually verify the web pages when running a GVO job
Change-Id: I3cf3f0da2be2be79e546762a8083d4de338ff6aa
Reviewed-on: http://gerrit.cloudera.org:8080/20428
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Add debug page for in-flight DDLs in catalogd
> ---------------------------------------------
>
> Key: IMPALA-9118
> URL: https://issues.apache.org/jira/browse/IMPALA-9118
> Project: IMPALA
> Issue Type: New Feature
> Components: Catalog
> Reporter: Quanlong Huang
> Assignee: Quanlong Huang
> Priority: Critical
> Labels: observability, supportability
> Attachments: Selection_082.png
>
>
> In a busy cluster, it's possible that many DDL/DML queries keep in the
> CREATED state for several minutes. Especially when using with sync_ddl=true,
> tens of minutes are also possible. They may be waiting for the ExecDdl RPC to
> catalogd to finish.
> It'd be helpful for debugging DDL/DML hangs if we can show the in-flight DDLs
> in catalogd. I think the following fields are important:
> * thread id
> * coordinator
> * db name / table name
> * ddl type, e.g. AddPartition, DropTable, CreateTable, etc. More types
> [here|https://github.com/apache/impala/blob/3.3.0/common/thrift/JniCatalog.thrift#L31].
> * last event, e.g. waiting for table lock, got table lock, loading file
> metadata, waiting for sync ddl version etc.
> * start time
> * time elapsed
> * (optional) params link to show the TDdlExecRequest in json format
> It'd be better to also include running REFRESH/INVALIDATE METADATA commands
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
