Shekharrajak commented on issue #19190: URL: https://github.com/apache/druid/issues/19190#issuecomment-4190906731
I'm very interested in supporting this Iceberg V2 integration! Regarding the memory pressure limitation in 3.1, I’d like to help implement a streaming iterator for the IcebergNativeRecordReader to avoid materializing all records into a List. Looking ahead, we should ensure this design is Iceberg V3-ready by considering: Puffin Files: Leveraging V3’s standard for stats/indexes to eventually speed up Druid’s metadata discovery. Field-ID Mapping: Ensuring our schema handling is robust enough for V3’s advanced evolution (like column renames/promotions) without re-indexing. Default Values: Preparing the IcebergRecordConverter to handle spec-level default values introduced in V3. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
