[
https://issues.apache.org/jira/browse/IMPALA-15331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yida Wu updated IMPALA-15331:
-----------------------------
Issue Type: Improvement (was: Bug)
> Queries fail due to catalog in standby mode in GetNullPartitionName
> -------------------------------------------------------------------
>
> Key: IMPALA-15331
> URL: https://issues.apache.org/jira/browse/IMPALA-15331
> Project: IMPALA
> Issue Type: Improvement
> Components: Frontend
> Reporter: Yida Wu
> Assignee: Yida Wu
> Priority: Major
>
> In IMPALA-15121, we fixed an issue where queries would fail during a Catalogd
> HA failover if an Impalad coordinator tried to fetch metadata from a
> restarted catalogd that was currently in standby mode.
> However, we are still seeing query failures during failovers with the
> following logging:
> {code:java}
> I20260831 23:48:12.869620 451880 jni-util.cc:335]
> de452853deebd4c8:9946356900000000] java.lang.RuntimeException:
> java.lang.RuntimeException: java.util.concurrent.ExecutionException:
> org.apache.impala.common.InternalException: Error requesting
> GetNullPartitionName: Request for Catalog service is rejected since catalogd
> quasar-rcprbj-2.vpc.eng.cloudera.com:26000 is in standby mode
> at
> org.apache.impala.service.Frontend.getTExecRequestWithFallback(Frontend.java:2493)
> at org.apache.impala.service.Frontend.createExecRequest(Frontend.java:2153)
> at
> org.apache.impala.service.JniFrontend.createExecRequest(JniFrontend.java:181)
> Caused by: java.lang.RuntimeException:
> java.util.concurrent.ExecutionException:
> org.apache.impala.common.InternalException: Error requesting
> GetNullPartitionName: Request for Catalog service is rejected since catalogd
> quasar-rcprbj-2.vpc.eng.cloudera.com:26000 is in standby mode
> at
> org.apache.impala.catalog.local.CatalogdMetaProvider.loadWithCaching(CatalogdMetaProvider.java:650)
> at
> org.apache.impala.catalog.local.CatalogdMetaProvider.loadNullPartitionKeyValue(CatalogdMetaProvider.java:1439)
> at
> org.apache.impala.catalog.local.LocalCatalog.getNullPartitionKeyValue(LocalCatalog.java:325)
> at
> org.apache.impala.catalog.local.LocalFsTable.getNullPartitionKeyValue(LocalFsTable.java:243)
> at
> org.apache.impala.catalog.local.LocalFsTable.toTHdfsTable(LocalFsTable.java:345)
> at
> org.apache.impala.catalog.local.LocalFsTable.toThriftDescriptor(LocalFsTable.java:318)
> at
> org.apache.impala.analysis.DescriptorTable.toThrift(DescriptorTable.java:238)
> at
> org.apache.impala.analysis.DescriptorTable.toSerializedThrift(DescriptorTable.java:259)
> at
> org.apache.impala.service.Frontend.getPlannedExecRequest(Frontend.java:3556)
> at
> org.apache.impala.service.Frontend.doCreateExecRequest(Frontend.java:3235)
> at org.apache.impala.service.Frontend.getTExecRequest(Frontend.java:2625)
> at
> org.apache.impala.service.Frontend.getTExecRequestWithFallback(Frontend.java:2470)
> ... 2 more
> Caused by: java.util.concurrent.ExecutionException:
> org.apache.impala.common.InternalException: Error requesting
> GetNullPartitionName: Request for Catalog service is rejected since catalogd
> quasar-rcprbj-2.vpc.eng.cloudera.com:26000 is in standby mode
> at
> java.base/java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:396)
> at
> java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2073)
> at
> com.google.common.util.concurrent.Uninterruptibles.getUninterruptibly(Uninterruptibles.java:246)
> at
> org.apache.impala.catalog.local.CatalogdMetaProvider.loadWithCaching(CatalogdMetaProvider.java:641)
> ... 13 more
> Caused by: org.apache.impala.common.InternalException: Error requesting
> GetNullPartitionName: Request for Catalog service is rejected since catalogd
> quasar-rcprbj-2.vpc.eng.cloudera.com:26000 is in standby mode
> at
> org.apache.impala.service.FeSupport.GetNullPartitionName(FeSupport.java:549)
> at
> org.apache.impala.catalog.local.CatalogdMetaProvider$9.call(CatalogdMetaProvider.java:1446)
> at
> org.apache.impala.catalog.local.CatalogdMetaProvider$9.call(CatalogdMetaProvider.java:1442)
> at
> org.apache.impala.catalog.local.CatalogdMetaProvider.loadWithCaching(CatalogdMetaProvider.java:625)
> ... 13 more
> I20260831 23:48:12.869776 451880 status.cc:123]
> de452853deebd4c8:9946356900000000] RuntimeException:
> java.lang.RuntimeException: java.util.concurrent.ExecutionException:
> org.apache.impala.common.InternalException: Error requesting
> GetNullPartitionName: Request for Catalog service is rejected since catalogd
> quasar-rcprbj-2.vpc.eng.cloudera.com:26000 is in standby mode
> CAUSED BY: RuntimeException: java.util.concurrent.ExecutionException:
> org.apache.impala.common.InternalException: Error requesting
> GetNullPartitionName: Request for Catalog service is rejected since catalogd
> quasar-rcprbj-2.vpc.eng.cloudera.com:26000 is in standby mode
> CAUSED BY: ExecutionException: org.apache.impala.common.InternalException:
> Error requesting GetNullPartitionName: Request for Catalog service is
> rejected since catalogd quasar-rcprbj-2.vpc.eng.cloudera.com:26000 is in
> standby mode
> CAUSED BY: InternalException: Error requesting GetNullPartitionName: Request
> for Catalog service is rejected since catalogd
> quasar-rcprbj-2.vpc.eng.cloudera.com:26000 is in standby mode
> {code}
> This error comes from a different code path for GetNullPartitionName than the
> previous issue which was on GetPartialCatalogObject. Probably we can
> introduce a similar retry idea to fix it, allowing the frontend to retry the
> fetch against the new active catalogd without failing the query.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
