GitHub user numinnex added a comment to the discussion: Arrow as zero-serialization target
I see that they support different layouts, but still how we will end up using this is most likely as an additional connector, e.g exposing an endpoint that allows any data lake to connect and read our data that is transformed to parquet/vortex, afaik redpanda topics with `Iceberg` feature turned on, using the tiered storage architecture, migrates the avro compatible schema messages to parquet and allows to directly read data in parquet format from the borker. GitHub link: https://github.com/apache/iggy/discussions/1602#discussioncomment-12478957 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
