nsivabalan commented on PR #18136: URL: https://github.com/apache/hudi/pull/18136#issuecomment-3892957270
> A better and general approach would be adding a file system view based on the latest snapshot only to limit the size of file slices in memory, which is used by the file index. That should solve the problem with better layering. Hey @yihua : based on latest state of the patch, I feel it nicely sits w/n HoodieTableMetadata and so, we can leverage this w/ any of FSV. Just that I see some special handling of FileIndex after filtering which we can avoid (shared feedback above). otherwise, current layering seems ok to me. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
