[
https://issues.apache.org/jira/browse/IMPALA-9006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Ho updated IMPALA-9006:
-------------------------------
Description: Currently, a Statestore subscriber starts a separate thread
after the initial registration with Statestore to periodically check if the
Statestore may have failed and re-registered with Statestore if necessary.
Similarly, the function {{StatestoreSubscriber::Register()}} also relies on the
old Thrift client's retry logic to retry failed RPC attempts to Statestore.
This is needed as the initial registration relies on this retry logic to wait
for Statestore to startup in case an Impala daemon starts before the
Statestore. These two retry paths may be consolidated. (was: Currently, a
Statestore subscriber starts a separate thread after the initial registration
with Statestore to periodically check if the Statestore may have failed and
re-registered with Statestore if necessary. Similarly, the function
{{StatestoreSubscriber::Register()}} also relies on the old Thrift client's
retry logic to retry failed RPC attempts to Statestore. This is needed as the
initial registration relies on this retry logic to wait for Statestore to
startup in case an Impala daemon starts before the Statestore. These two retry
paths may be consolidated.
Last but not least, the current registration logic at Statestore doesn't check
if the address provided by the subscriber can actually be resolved. In certain
deployment scenarios, it's possible that the address passed by a subscriber is
not yet resolvable (e.g. a Kubernetes pod whose readiness probe failed).
Statestore should check for if the address is resolvable and fail the
registration if not. The subscriber can keep retrying until its address can be
resolved by Statestore. This is particularly useful in configuration where
readiness probe of a pod in Kubernetes is exploited for a warm backup
configuration.)
> Consolidate the Statestore subscriber's retry logic
> ---------------------------------------------------
>
> Key: IMPALA-9006
> URL: https://issues.apache.org/jira/browse/IMPALA-9006
> Project: IMPALA
> Issue Type: Improvement
> Components: Distributed Exec
> Affects Versions: Impala 3.4.0
> Reporter: Michael Ho
> Assignee: Michael Ho
> Priority: Major
>
> Currently, a Statestore subscriber starts a separate thread after the initial
> registration with Statestore to periodically check if the Statestore may have
> failed and re-registered with Statestore if necessary. Similarly, the
> function {{StatestoreSubscriber::Register()}} also relies on the old Thrift
> client's retry logic to retry failed RPC attempts to Statestore. This is
> needed as the initial registration relies on this retry logic to wait for
> Statestore to startup in case an Impala daemon starts before the Statestore.
> These two retry paths may be consolidated.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
