[
https://issues.apache.org/jira/browse/IMPALA-14107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17956628#comment-17956628
]
Quanlong Huang commented on IMPALA-14107:
-----------------------------------------
Might not be related to this issue, but the query id in the logs of
GetPartialCatalogObject RPCs is missleading since it's actually the query id of
the previous RPC that uses the same thread. We should pass the query id in the
GetPartialCatalogObject request and set it in the thread debug info here:
[https://github.com/apache/impala/blob/d630d6f8af8cd86a845fc0415c99b8aa6608c28f/be/src/catalog/catalog.cc#L117-L120]
like other catalog RPCs:
{code:cpp}
Status Catalog::ExecDdl(const TDdlExecRequest& req, TDdlExecResponse* resp) {
if (req.__isset.header && req.header.__isset.query_id) {
GetThreadDebugInfo()->SetQueryId(req.header.query_id); // Needs this
}
return JniUtil::CallJniMethod(catalog_, exec_ddl_id_, req, resp);
}{code}
[https://github.com/apache/impala/blob/d630d6f8af8cd86a845fc0415c99b8aa6608c28f/be/src/catalog/catalog.cc#L139-L143]
[~MikaelSmith]'s patch adds the query id by the way and can be used to fix the
query ids in catalogd logs:
[https://gerrit.cloudera.org/c/22738/3/common/thrift/CatalogService.thrift]
> test_reload_events_with_transient_partitions stuck in local catalog mode
> ------------------------------------------------------------------------
>
> Key: IMPALA-14107
> URL: https://issues.apache.org/jira/browse/IMPALA-14107
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
> Reporter: Riza Suminto
> Assignee: Riza Suminto
> Priority: Major
> Attachments: catalogd.INFO, impalad.INFO
>
>
> test_reload_events_with_transient_partitions can get stuck when running in
> local catalog mode. From the catalod.INFO, it looks like catalog keep looking
> for non-existent partition id.
> {noformat}
> W20250527 16:04:11.311885 1240999 HdfsTable.java:2302]
> 5b44f2b8f845ceb7:2febe44200000000] Missing partition ID: 59, Table:
> test_reload_events_with_transient_partitions_local_catalog_77dd977e.tbl
> W20250527 16:04:12.118695 1240999 HdfsTable.java:2302]
> 5b44f2b8f845ceb7:2febe44200000000] Missing partition ID: 59, Table:
> test_reload_events_with_transient_partitions_local_catalog_77dd977e.tbl
> W20250527 16:04:13.121974 1240999 HdfsTable.java:2302]
> 5b44f2b8f845ceb7:2febe44200000000] Missing partition ID: 59, Table:
> test_reload_events_with_transient_partitions_local_catalog_77dd977e.tbl
> W20250527 16:04:14.325909 1240999 HdfsTable.java:2302]
> 5b44f2b8f845ceb7:2febe44200000000] Missing partition ID: 59, Table:
> test_reload_events_with_transient_partitions_local_catalog_77dd977e.tbl
> ...
> {noformat}
> Attached are both impalad and catalogd log when the issue happen.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
