[
https://issues.apache.org/jira/browse/IMPALA-12323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17748744#comment-17748744
]
Wenzhe Zhou commented on IMPALA-12323:
--------------------------------------
Tried different cases, and found the DDL query does not hang if only one
catalogd is alive when CatalogD HA is enabled. For example, start cluster with
only one catalogd and CatalogD HA is enabled, DDL does not hang. Start cluster
with two catalogd and CatalogD HA is enabled, then kill primary catalogd or
standby catalogd, DDL does not hang.
There was following log messages in the coordinator log file:
{code:java}
I0728 07:12:59.567205 1283370 client-request-state.cc:774]
714e19dddfad44ea:32bd59c800000000] DDL exec mode=asynchronous
I0728 07:12:59.600250 1283373 impala-server.cc:2197]
714e19dddfad44ea:32bd59c800000000] Waiting for catalog version: 0 current
version: 1670
I0728 07:12:59.600257 1283373 impala-server.cc:2207]
714e19dddfad44ea:32bd59c800000000] Received catalog version: 0
I0728 07:12:59.600258 1283373 impala-server.cc:2216]
714e19dddfad44ea:32bd59c800000000] Waiting for min subscriber topic version:
1668 current version: 0
{code}
min_subscriber_topic_version of the catalog topic does not increase due to the
standby catalogd.
> DDL hang with SYNC_DDL=1 when CatalogD HA enabled
> -------------------------------------------------
>
> Key: IMPALA-12323
> URL: https://issues.apache.org/jira/browse/IMPALA-12323
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Affects Versions: Impala 4.3.0
> Reporter: Wenzhe Zhou
> Assignee: Wenzhe Zhou
> Priority: Critical
> Fix For: Impala 4.3.0
>
>
> When CatalogD HA is enabled, standby catalogd does not receive catalog topic
> update from statestore and does not apply catalog updates from the active
> catalogd. Its min topic version is not changed.
> Function Statestore::GetMinSubscriberTopicVersion() loops through all
> subscribers including standby catalogd to find min topic version. Standby
> catalogd causes min_subscriber_topic_version not increased, and hence Impala
> server waits indefinitely in
> ImpalaServer::WaitForCatalogUpdateTopicPropagation().
> To fix the issue, skip standby catalogd when finding min topic version in
> Statestore::GetMinSubscriberTopicVersion().
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
