SteNicholas opened a new pull request, #580: URL: https://github.com/apache/paimon-cpp/pull/580
### Purpose Linked issue: close #578 Support GEOMETRY and GEOGRAPHY columns in Paimon C++, following Paimon Java. Values remain OGC Well-Known Binary (WKB) in Arrow `Binary` arrays, with the logical type, coordinate reference system (CRS) and geography edge interpolation algorithm preserved through schema and data round trips. - Add `FieldType::GEOMETRY` and `FieldType::GEOGRAPHY`, and parse and serialize their schema declarations, including quoted CRS parameters, nullability and nested ROW, ARRAY and MAP fields. Match Java's default `OGC:CRS84` CRS, case-insensitive CRS comparison and five geography edge algorithms. - Preserve `geoarrow.wkb` field metadata, field IDs and nullability through the Arrow C Data Interface, nested projections, field mapping, row tracking, data evolution and MAP shredding. Preserve declared MAP key nullability in table schemas while keeping stored keys non-null. - Make type comparison, casts and schema evolution distinguish BYTES, GEOMETRY, GEOGRAPHY, different CRS values and different edge algorithms. Reject geospatial primary, partition and bucket keys, sequence fields and sequence-group ordering keys, and clustering/sort columns. - Backport native Parquet GEOMETRY/GEOGRAPHY annotations to Arrow 17 while retaining C++17. Store WKB in BYTE_ARRAY columns and preserve the annotations and parameters for nested columns, including Java-compatible LIST element and MAP key/value field IDs. - Require Parquet data and changelog files, including per-level formats and operation-specific overrides, and reject geospatial schemas at unsupported format readers and writers. - Retain null counts while suppressing WKB min/max statistics in Paimon, the realtime store and Parquet. Ignore WKB bounds in file, row-group and page pruning, treat incomplete null counts as unknown, and support null predicates without changing ordinary BYTES or partition predicate behavior. Support is initially limited to Parquet. Spatial functions, coordinate transformations and spatial indexes are outside this change. ### Tests - Schema and type tests for default and custom CRS values, quoted and Unicode CRS parameters, all five geography edge algorithms, invalid metadata, logical equality, incompatible casts and schema evolution, and nested C Data Interface round trips. - Metadata preservation tests for projections, field mapping, MAP shredding, data evolution, row tracking and BLOB view resolution; validation tests for unsupported key/order usages and file formats, including per-level and operation-specific overrides. - Statistics and predicate tests for retained null counts, absent geospatial min/max bounds, incomplete row-group statistics, null predicates, rejected comparisons, page pruning and unchanged BYTES behavior. - Parquet tests for native annotations and field IDs, nested ROW/ARRAY/MAP values, nulls and valid empty geometries, sliced array adaptation, incompatible read schemas, default native parameters and escaped CRS round trips. - Integration tests for table reads with reordered projections and null predicates, and Java-to-C++ and C++-to-Java Parquet interoperability covering WKB values, logical parameters, nested annotations and statistics. ### API and Format - API: adds `FieldType::GEOMETRY` and `FieldType::GEOGRAPHY` in `include/paimon/defs.h`. `Schema::GetFieldType` identifies these types from Arrow field metadata. - Format: table schemas preserve GEOMETRY/GEOGRAPHY declarations and parameters. Arrow fields carry `geoarrow.wkb` metadata, and Parquet BYTE_ARRAY columns carry native GEOMETRY/GEOGRAPHY logical annotations compatible with Paimon Java. Existing BYTES columns retain their binary semantics. ### Documentation Yes. `docs/source/user_guide/data_types.rst` documents the geospatial schema syntax, CRS and edge algorithms, Arrow metadata, Parquet storage, nested fields and MAP key nullability, key/order and cast restrictions, statistics and predicates, and the Java/C++ interoperability test. ### Generative AI tooling Generated-by: OpenAI Codex (GPT-6) 🤖 Generated with [OpenAI Codex](https://openai.com/codex/) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
