OIiveirra opened a new pull request, #67907:
URL: https://github.com/apache/doris/pull/67907
### What problem does this PR solve?
Issue Number: N/A (DORIS-28228)
Related PR: N/A
Problem Summary:
Add Iceberg V3 `GEOMETRY` and `GEOGRAPHY` support for external tables. The
change adds bidirectional Iceberg schema mapping with CRS and Geography
algorithm preservation, Arrow/Parquet spatial annotations, raw WKB read/write
handling, and spatial function support.
Scope is limited to Iceberg external tables. Doris internal tables reject
these types. Paimon, Lance, GeoArrow, spatial indexes, bounds pruning, and
predicate pushdown are intentionally excluded.
For Geography, non-default CRS/algorithm values are preserved for read and
write. Spatial measurement and relationship functions currently require
`GEOGRAPHY(OGC:CRS84, spherical)` and return a stable NotSupported error for
other combinations to avoid incorrect calculations.
### Release note
Support Iceberg V3 GEOMETRY and GEOGRAPHY external-table schema mapping and
raw-WKB Parquet read/write.
### Check List (For Author)
- Test
- [ ] Regression test
- [x] Unit Test
- FE targeted tests: 15 passed (`IcebergUtilsTest`,
`IcebergSpatialWriteAnalyzerTest`, `SpatialConstructorFunctionTest`, and
`ColumnDefinitionTest`).
- BE spatial/Iceberg production and test objects compiled
successfully after rebase.
- Iceberg spatial regression Groovy script compiled successfully.
- The full external-table regression requires Iceberg REST, MinIO,
and Spark services and was not run locally.
- A clean FE rebuild on the latest upstream baseline is currently
blocked before this PR code is compiled by upstream
`javax.annotation.Generated` and Nereids pattern-generator classpath issues.
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- Behavior changed:
- [ ] No.
- [x] Yes. Iceberg external tables can read and write Iceberg V3
GEOMETRY/GEOGRAPHY values as raw WKB; unsupported WKB metadata and unsupported
Geography computation semantics fail explicitly.
- Does this need documentation?
- [ ] No.
- [x] Yes. A website documentation update will follow once the
multi-algorithm Geography computation scope is finalized.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR should
merge into -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
