JingsongLi opened a new pull request, #9251: URL: https://github.com/apache/paimon/pull/9251
## What changed - Add `GEOMETRY` and `GEOGRAPHY` data types aligned with Iceberg, including CRS and geography edge algorithms. - Store geospatial values as WKB and integrate them with Paimon row data, serializers, schema JSON, statistics, Arrow, and Parquet. - Preserve geospatial logical annotations in Iceberg metadata and native Parquet schemas. - Add Python type, casting, schema, writer, and documentation support. - Add native Spark 4.1 SQL support and an explicit compatibility boundary for older Spark versions. - Expose geospatial values as `VARBINARY` in Flink SQL while preserving the Paimon geospatial schema. ## Why Iceberg now supports geospatial logical types. Paimon needs compatible schema semantics and storage behavior so geometry and geography columns can be represented and round-tripped across the Java, Python, Iceberg, Parquet, Spark, and Flink integrations. ## User impact Users can define geometry and geography columns in Paimon schemas, persist WKB values, retain CRS and edge-algorithm metadata, and use the types natively through Spark 4.1. Flink SQL exposes the values as binary because the supported Flink versions do not provide a native geospatial logical type. ## Validation - Focused Java suites: 351 tests passed. - Core `TableTestBase` geospatial table test passed. - Spark 4.1 native Java tests: 2 passed. - Spark 4.1 SQL geospatial test passed. - Flink SQL geospatial ITCase passed with both Flink 1.x and Flink 2.x profiles. - Python geospatial tests passed, including 32 tests and 11 subtests in the related suites. - Spotless formatting and `git diff --check` passed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
