[
https://issues.apache.org/jira/browse/HUDI-5348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ethan Guo updated HUDI-5348:
----------------------------
Description: As of now, we only cache the log file reader. We should also
cache the latest file slices at partition level since we keep calling
getLatestFileSlices often. This causes a new file system view to be
instantiated each time and thus lists the {{files}} partition. The FS listing
call can be avoided by caching the file system view inside
{{HoodieBackedTableMetadata}} . (was: As of now, we only cache the base file
reader and log file reader. We should also cache the latest file slices at
partition level since we keep calling getLatestFileSlices often. This causes a
new file system view to be instantiated each time and thus lists the {{files}}
partition. The FS listing call can be avoided by caching the file system view
inside {{HoodieBackedTableMetadata}} .)
> Cache file slices within MDT reader
> -----------------------------------
>
> Key: HUDI-5348
> URL: https://issues.apache.org/jira/browse/HUDI-5348
> Project: Apache Hudi
> Issue Type: Improvement
> Reporter: Ethan Guo
> Assignee: Ethan Guo
> Priority: Blocker
> Fix For: 0.12.2
>
>
> As of now, we only cache the log file reader. We should also cache the latest
> file slices at partition level since we keep calling getLatestFileSlices
> often. This causes a new file system view to be instantiated each time and
> thus lists the {{files}} partition. The FS listing call can be avoided by
> caching the file system view inside {{HoodieBackedTableMetadata}} .
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
