c-thiel commented on issue #1797: URL: https://github.com/apache/iceberg-rust/issues/1797#issuecomment-3466541151
We just maintain a very leightweight fork for Lakekeeper to be able to move quicker with unmerged / unreleased PRs. We also require access to some methods considered internal, such as building unsafe TableMetadata from its parts or access to `ManifestEntry.inherit_data`. My biggest painpoint with iceberg-rust is its FileIO not beeing Trait-based and not returning specific errors (i.e. a matchable file not found). In FileIO not all auth mechanisms are supported, and I don't think it is useful to re-implement all different IRSA / IMDS mechanisms of all hyperscalers. We are also missing refresh mechanisms for very long running tasks. Because of this, at Lakekeeper, we are not using Iceberg Rust FIleIO at all but instead use the hyperscaler frameworks behind a trait. Exactly this trait based approach is discussed for iceberg-rust as well in [this proposal](https://docs.google.com/document/d/1-CEvRvb52vPTDLnzwJRBx5KLpej7oSlTu_rg0qKEGZ8/edit?tab=t.dgr4vjtmzh92) Finally I will try to do more reviews again in the future. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
