[
https://issues.apache.org/jira/browse/IMPALA-12060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17718367#comment-17718367
]
Wenzhe Zhou commented on IMPALA-12060:
--------------------------------------
Submitted a patch: https://gerrit.cloudera.org/#/c/19826/
Currently the DNS resolution are done on subscriber's side, not on statestore.
Subscribers (coordinator/executor/catalogd) do DNS resolution if
"FLAGS_statestore_subscriber_use_resolved_address" is set as true, then send
the resolved heart-beat address to statestore. statestore don't have host-name
of the subscribers and cannot do DN resolution now.
In the patch, the DNS resolution are still done on subscriber's side. But each
subscriber need to re-do DNS resolution when it try to re-register to
statestore in recovery mode. If DNS resolution fails, it will not re-register
itself to statestore. This prevents the catalogd in passive mode (original
leader) from rejoining the cluster. When catalogd leader failover happens, new
leader catalogd could do DNS resolution, and send registration to statestore.
Then statestore un-register original leader catalogd before register new leader
catalogd to ensure only one catalogd instance in the cluster.
Another approach is to move DNS resolution from subscriber to statestore so
that statestore have host-name of the subscribers and could do DNS resolution.
statestore will try to do DNS resolution with hostname of the subscribers when
it try to register a subscriber. If DNS resolution fails, statestore will
reject the registration request.
> statestore should only allow a single catalog instance to be part of cluster
> ----------------------------------------------------------------------------
>
> Key: IMPALA-12060
> URL: https://issues.apache.org/jira/browse/IMPALA-12060
> Project: IMPALA
> Issue Type: Bug
> Reporter: Abhishek Rawat
> Assignee: Wenzhe Zhou
> Priority: Critical
>
> Today Impala doesn't support multiple catalogd instances to be part of the
> cluster. When using external Active Passive HA mechanisms like leader
> election, it may be possible for multiple catalog replicas to be part of the
> cluster, in some cases. This is mainly because catalogds have no idea that
> there are other catalogd instances. One way to address this issue could be
> for statestore to always ensure that there could only be a single instance of
> catalogd in the cluster. Coordinator also has a config `catalog_service_host`
> - both statestore and coordinator should have a consistent view and consider
> the proper catalogd instance to be part of the cluster.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
