bvaradar commented on PR #10419: URL: https://github.com/apache/hudi/pull/10419#issuecomment-1870684299
@majian1998 ``` After PR https://github.com/apache/hudi/pull/7871, Hudi would recreate the relation with each query, obtaining the latest snapshot. Therefore, I believe there should be an assessment at the start of listFiles to determine whether the cache (snapshot) needs to be updated. ```` Can you check if HoodieFileIndex is used in other places apart from spark sql execution. Also, if the relation is getting recreated with each query, wouldn't the file-index be force-refreshed ? Not sure if I am following your comment correctly. As the check has the cost of updating the timeline, wondering if this is needed, should we explicitly perform the check at the caller side when we know we need to refresh ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
