[
https://issues.apache.org/jira/browse/IMPALA-14695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Quanlong Huang updated IMPALA-14695:
------------------------------------
Attachment: profile_retry_example.txt
> Fast path for simple partition queries to fetch missing metadata from
> catalogd in batch
> ---------------------------------------------------------------------------------------
>
> Key: IMPALA-14695
> URL: https://issues.apache.org/jira/browse/IMPALA-14695
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog, Frontend
> Reporter: Quanlong Huang
> Assignee: Quanlong Huang
> Priority: Critical
> Attachments: profile_retry_example.txt
>
>
> It's a regression of local catalog mode (i.e. catalog-v2) that query planning
> needs to retry when metadata changes in catalogd side. Example coordinator
> logs:
> {code:java}
> W0121 11:23:13.707223 317257 CatalogdMetaProvider.java:469]
> 0f4d14a4e9abc477:7cfdb09400000000] Catalog object TCatalogObject(type:TABLE,
> catalog_version:19032, table:TTable(db_name:mydb, tbl_name:mytbl)) changed
> version from 19032 to 19041 while fetching metadata
> W0121 11:23:20.107473 317257 Frontend.java:2127]
> 0f4d14a4e9abc477:7cfdb09400000000] Retrying plan of query alter table
> mydb.mytbl partition (p='2268357') set tblproperties('numRows'='1325',
> 'STATS_GENERATED_VIA_STATS_TASK'='true'): Catalog object
> TCatalogObject(type:TABLE, catalog_version:19032, table:TTable(db_name:mydb,
> tbl_name:mytbl)) changed version between accesses. (retry #32 of 40)
> {code}
> For simple queries like REFRESH PARTITION, COMPUTE INCREMENTAL STATS on a
> single partition, or ALTER TABLE on a single partition, if some metadata is
> missing in coordinator's local cache, we can consider sending a single batch
> request to catalogd to fetch all the metadata the query needs, thus to avoid
> hitting InconsistentMetadataFetchException which requires retries.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
