Yida Wu created IMPALA-15121:
--------------------------------
Summary: Queries fail with "catalogd is in standby mode" during HA
failover
Key: IMPALA-15121
URL: https://issues.apache.org/jira/browse/IMPALA-15121
Project: IMPALA
Issue Type: Improvement
Components: Frontend
Affects Versions: Impala 4.5.0
Reporter: Yida Wu
When Catalogd HA is enabled, queries can fail during a catalogd failover with
the following error:
{code:java}
java.lang.RuntimeException:
org.apache.impala.catalog.local.LocalCatalogException: Unable to load database
names
CAUSED BY: TException: Request for Catalog service is rejected since catalogd
<host>:<port> is in standby mode
{code}
Reproduce steps:
1. Kill the active catalogd.
2. Run a query that needs to access the catalog:
{code:java}
refresh functional.alltypes;
{code}
3. Restart the killed catalogd.
The root cause is that when the catalogd is down, the frontend of impalad will
keep trying to do RPCs with the current active catalogd, until it eventually
comes up again. Because it comes back in standby status, the frontend will
receive the standby rejection error introduced in IMPALA-12849, which ends up
as a query failure. We should retry when receiving this error on the impalad
side so that it can wait to receive the statestore update and talk to the
correct active catalogd without a query failure.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
