huan233usc commented on PR #16851: URL: https://github.com/apache/iceberg/pull/16851#issuecomment-4783450714
Agreed on aligning to explicit defaults (matching #16765 — "write out the crs/algorithm for clarity"). Concretely how I'd reconcile the two PRs: - **CRS — keep explicit end to end, don't nullify on default.** #16765 now persists `geometry(OGC:CRS84)` / `geography(OGC:CRS84, spherical)` in table metadata and writes the resolved CRS into the Parquet annotation, instead of collapsing it when it equals the default. Once this PR rebases on #16765, the Iceberg side stays explicit; on the Spark→Iceberg path `Types.GeometryType.of(crs)` / `GeographyType.of(crs)` resolves through the same getters, so it serializes explicitly too. So the only place a default still "disappears" is inside Spark's own type, which is out of our control. - **Edge algorithm — also explicit on the Iceberg side; reject (not coerce) at the Spark boundary.** #16765 writes `spherical` explicitly rather than dropping it. Spark's `GeometryType`/`GeographyType` has no slot for the edge algorithm, so non-spherical geography genuinely can't round-trip through Spark — rejecting it (as in 2f23994) is the safe choice rather than silently coercing to spherical. So the only thing lost at the Spark boundary is non-spherical geography, and it's an explicit failure, not a silent one. Net: defaults are written explicitly (CRS + algorithm) everywhere they can be represented, and the Spark boundary only refuses what Spark genuinely can't model. Does that match what you had in mind for the metadata passthrough? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
