[
https://issues.apache.org/jira/browse/IMPALA-12323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17748773#comment-17748773
]
ASF subversion and git services commented on IMPALA-12323:
----------------------------------------------------------
Commit 6ecb8bfcf4d9e68e0091aea64540e0fb64aeb3e0 in impala's branch
refs/heads/master from wzhou-code
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=6ecb8bfcf ]
IMPALA-12323: DDL hang with SYNC_DDL=1 when CatalogD HA enabled
When CatalogD HA is enabled, standby catalogd does not receive catalog
topic updates from statestore and does not apply catalog updates from
the active catalogd. Its min topic version is not changed.
Function Statestore::GetMinSubscriberTopicVersion() loops through all
subscribers to find min topic version. Standby catalogd causes
min topic version not increased, and hence Impala server waits
indefinitely in ImpalaServer::WaitForCatalogUpdateTopicPropagation().
This patch fixed the issue by skipping standby catalogd when finding min
topic version in Statestore::GetMinSubscriberTopicVersion().
Testing:
- Added unit-test code for CatalogD HA to run DDL with SYNC_DDL as 1.
Verified that test cases hang without fix, and test cases were passed
after fix.
- Passed test_catalogd_ha.py.
Change-Id: Ie559c711078f32171dfb2d2e2fda54773c0927c3
Reviewed-on: http://gerrit.cloudera.org:8080/20280
Reviewed-by: Andrew Sherman <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> DDL hang with SYNC_DDL=1 when CatalogD HA enabled
> -------------------------------------------------
>
> Key: IMPALA-12323
> URL: https://issues.apache.org/jira/browse/IMPALA-12323
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Affects Versions: Impala 4.3.0
> Reporter: Wenzhe Zhou
> Assignee: Wenzhe Zhou
> Priority: Critical
> Fix For: Impala 4.3.0
>
>
> When CatalogD HA is enabled, standby catalogd does not receive catalog topic
> update from statestore and does not apply catalog updates from the active
> catalogd. Its min topic version is not changed.
> Function Statestore::GetMinSubscriberTopicVersion() loops through all
> subscribers including standby catalogd to find min topic version. Standby
> catalogd causes min_subscriber_topic_version not increased, and hence Impala
> server waits indefinitely in
> ImpalaServer::WaitForCatalogUpdateTopicPropagation().
> To fix the issue, skip standby catalogd when finding min topic version in
> Statestore::GetMinSubscriberTopicVersion().
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
