parthchandra commented on issue #1040: URL: https://github.com/apache/datafusion-comet/issues/1040#issuecomment-2447977412
> * Support for complex types. The parquet crate already supports reading maps and structs. @andygrove The doc here https://crates.io/crates/parquet says the reader supports ` Primitive column value readers`. Doesn't say whether complex types are supported. Also, something to keep in mind, Comet has Spark specific decoding (for instance Int96 timestamps https://github.com/apache/datafusion-comet/blob/00cf79b253d9fa31dcc0506facbaf7b2dac25cdd/native/core/src/parquet/read/values.rs#L841). Some post processing may need to be done unless we can hook in our own decoder for special cases like this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
