[
https://issues.apache.org/jira/browse/IMPALA-7538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17129231#comment-17129231
]
Quanlong Huang commented on IMPALA-7538:
----------------------------------------
I did a quick investigation on this. The most missing block is
LocalCatalog.getHdfsCachePool(String). Coordinators use this to analyze DDL
statements to make sure the given cache pool exists. The DDLs are executed in
catalogd so we already have the functionality.
Only the cache pool names are propagated to coordinators. For cache replica
locations, they are in the FileDescriptors which we already have in partition
metadata.
So I think we can implement this by:
* Sync cache pool names to CatalogdMetaProvider as what we do in catalog-v1.
* DirectMetaProvider fetches the cache pool names from HDFS directly.
* Don't need to introduce FeFsCachePool. Because if we do so and derive
HdfsCachePool and LocalFsCachePool from it, there will be no difference between
HdfsCachePool and LocalFsCachePool.
cc [~vihangk1], [[email protected]]
> Support HDFS caching with LocalCatalog
> --------------------------------------
>
> Key: IMPALA-7538
> URL: https://issues.apache.org/jira/browse/IMPALA-7538
> Project: IMPALA
> Issue Type: Sub-task
> Reporter: Todd Lipcon
> Assignee: Quanlong Huang
> Priority: Major
> Labels: catalog-v2
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
