[
https://issues.apache.org/jira/browse/IMPALA-10801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Quanlong Huang updated IMPALA-10801:
------------------------------------
Fix Version/s: Impala 4.1.0
> Check the latest compaction Id before serving request
> -----------------------------------------------------
>
> Key: IMPALA-10801
> URL: https://issues.apache.org/jira/browse/IMPALA-10801
> Project: IMPALA
> Issue Type: Improvement
> Components: Catalog
> Reporter: Yu-Wen Lai
> Assignee: Yu-Wen Lai
> Priority: Major
> Fix For: Impala 4.1.0
>
>
> Cache compaction Id for a given table/file-metadata in CatalogD.
> Whenever there is a read request to CatalogD, get the latest compaction event
> Id from HMS, compare it with what is cached in CatalogD, and based on that
> decide whether to serve the data from cache or to refresh it from the
> filesystem. This can avoid notification based cache invalidation.
> Also, since there will be an open txn for the current long running query
> which is being served from CatalogD, we can be sure that current
> file-metadata being served is not already deleted by the cleaner.
> This proposal will use a new HMS API
> (https://issues.apache.org/jira/browse/HIVE-24828) to get the latest
> compaction id for a table.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
