GitHub user parthchandra added a comment to the discussion: Fusion of Iceberg Rust, Datafusion and Comet
It is worth reviving this thread based on Matt's PR above. IMHO, this is looking like the ideal way forward for Comet+Iceberg. The current integration using the `native_comet` implementation is complicated by the fact that Parquet libraries are shaded in Iceberg and the integration points between Comet and Iceberg all have Parquet classes in the APIs. Additionally, the `native_iceberg_compat` implementation is difficult to integrate into the current Iceberg implementation because it does not fit into callback based parquet reader creation pattern. GitHub link: https://github.com/apache/datafusion-comet/discussions/1031#discussioncomment-14639723 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
