C-Loftus commented on issue #989: URL: https://github.com/apache/iceberg-go/issues/989#issuecomment-6642688353
Question @laskoviymishka / @zeroshade , correct me if I am wrong but I am pretty sure at the moment the parquet writer in arrow-go that iceberg-go uses doesn't write geospatial row group statistics. I would like to add this. However, I was curious if you have any preference on where this belongs? To my understanding, I can't implement this in geoarrow-go since the only hooks arrow-go exposes into the writer is `ExtensionCustomParquetType.ParquetLogicalType()` (`parquet/pqarrow/schema.go:128`), which chooses the logical type and nothing more. Also the chunk metadata builder's thrift struct is unexported, so an extension type has no way to contribute column-chunk metadata. Adding a new custom hook to compute these statistics and contribute the metadata into the writer would also be an arrow-go change anyways, and it would diverge from the C++ and Rust design where the writer computes statistics from the logical type alone so I'm inclined not to do that. However, if I contribute this to arrow-go directly then arrow-go is likely to either require pulling in the go-geom dependency for working with WKB or reimplement a fair bit of WKB parsing. Did either of you have thoughts on the best way to approach this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
