[
https://issues.apache.org/jira/browse/IMPALA-12321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17748774#comment-17748774
]
ASF subversion and git services commented on IMPALA-12321:
----------------------------------------------------------
Commit e48af8c04ade5b270b1c601b8904fbd144241e29 in impala's branch
refs/heads/master from wzhou-code
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=e48af8c04 ]
IMPALA-12321: Fix the race condition when updating active catalogd
When CatalogD HA is enabled, statestored sends the address of current
active catalogd to coordinators and catalogds on two different events:
catalogd registration, and new active catalogd election change.
Statestored sends the active catalogd in two different kinds of RPCs.
If there are more than one election changes in short time, coordinators
and catalogds could receive RPCs in the order which are different from
the changing order on statestore.
To make coordinators and catalogds to have same view as statestore, we
have to avoid to overwrite the latest version of active catalogd with
previous change.
Version of active catalogd is already in UpdateCatalogd RPC, but not
in response message of statestore registration. This patch adds active
catalogd version in the response message of statestore registration.
Coordinators and catalogds only apply the changes which have newer
version than the version of last recevied active catalogd.
The version of last received active catalogd have to be re-synced for
new registration since statestore could be restarted.
Allow subscribers to skip UpdateCatalogd RPC if subscribers cannot
handle it, for example the statestore-id is unknown for a subscriber
since the RPC is received before registration is completed. The
skipped RPCs will be resent by statestore.
Testing:
- Added a test case to start both catalogds with flag
'force_catalogd_active' as true.
- Passed core tests
Change-Id: Ie49947e563d43c59bdd476b28c35be69848ae12a
Reviewed-on: http://gerrit.cloudera.org:8080/20276
Tested-by: Impala Public Jenkins <[email protected]>
Reviewed-by: Wenzhe Zhou <[email protected]>
> Fix the race condition for updating active catalogd when CatalogD HA is
> enabled
> -------------------------------------------------------------------------------
>
> Key: IMPALA-12321
> URL: https://issues.apache.org/jira/browse/IMPALA-12321
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
> Affects Versions: Impala 4.3.0
> Reporter: Wenzhe Zhou
> Assignee: Wenzhe Zhou
> Priority: Major
> Fix For: Impala 4.3.0
>
>
> When CatalogD HA is enabled, statestored sends the address of current active
> catalogd to coordinators and catalogds during registration, or a new active
> catalogd is elected. Statestored sends the active catalogd in two different
> kinds of RPCs. If there are more than one election changes in short time,
> coordinators and catalogds could receive RPCs in the order which is different
> from the changing order on statestore. To make coordinators and catalogds to
> have same view as statestore, we have to avoid to overwrite the later version
> of active catalogd with previous change.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
