C-Loftus opened a new issue, #2193: URL: https://github.com/apache/iceberg-go/issues/2193
Making this issue as a follow up to https://github.com/apache/iceberg-go/pull/2125 based on this comment: https://github.com/apache/iceberg-go/pull/2125#pullrequestreview-5502809355 Previously, when iceberg-go read a Parquet file with no embedded ARROW:schema metadata, it converted the Parquet schema to Arrow using the default mapping: a Parquet GEOMETRY/GEOGRAPHY logical type fell through to plain binary (since the physical type is BYTE_ARRAY). After the bump: https://github.com/geoarrow/geoarrow-go/pull/6 which we have after merge of https://github.com/apache/iceberg-go/pull/2125 adds ArrowTypeFromParquet, which recognizes the Parquet geo logical types and maps them to the geoarrow.wkb extension type. iceberg-go then sees the file column as geometry, not binary. The bump changes the outcome for the same file depending on the table’s declared type. For a v3 GEOMETRY(crs=srid:4326) column, a file with no ARROW:schema used to fail with cannot promote binary to geometry(srid:4326) and now reads cleanly, which is a fix. But a table that declares the column as binary (e.g. a v2 table populated via AddFiles from native-geo GeoParquet) used to work and now fails with cannot promote geometry to binary, and the AddFiles compat check flips the same way. There is no explicit geometry to binary rule. Whether a WKB column should project onto a binary column should be decided on. Here are the two options, I'd be interested to hear others' opinions on this: *Allow it*: the physical bytes are WKB in a BYTE_ARRAY either way, so reading geometry as binary is lossless, and it preserves backward compatibility for existing v2 tables. *Disallow it*: Iceberg treats geometry and binary as distinct logical types, and silently accepting the mismatch could hide a schema error. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
