Xuanwo commented on issue #1217: URL: https://github.com/apache/iceberg-rust/issues/1217#issuecomment-2809902851
> I think adding dynamic cache layer is one option, but not ideal. I think a better chioce maybe putting caching layer in other components like data file reader, which could do fine grain control. For example, a parquet reader could even choose to cache some columns, rather whole file. Yes, I believe there are several levels of caching that can occur at different layers. @sundy-li is referring to a transparent cache within FileIO (as in opendal), while @liurenjie1024 is talking about an explicit cache outside of FileIO (such as in FileReader). I think both are necessary to make iceberg-rust the best iceberg implementation. The most important thing to me is to implement them in an extensible way, allowing engines to have full control to optimize them. I'm willing to help implementtion both of them. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
