nevzheng opened a new issue, #11955:
URL: https://github.com/apache/gravitino/issues/11955
### Describe the feature
Add native `Types.GeometryType(crs)` and `Types.GeographyType(crs,
algorithm)` to Gravitino's unified type model so Iceberg V3 geospatial columns
load through the native metadata API as first-class types instead of today's
read-only `ExternalType("GEOMETRY")` / `ExternalType("GEOGRAPHY")`.
Next in the V3 net-new type series after native `variant` (#11927 / #11932 /
#11949) and `unknown → NullType` (#11951). Both are parameterized
`PrimitiveType`s (cloned from `FixedType`/`DecimalType`), so unlike `variant`
they carry parameters, and unlike `unknown` there's no existing type to reuse.
### Motivation
Iceberg V3 adds `geometry(crs)` (planar) and `geography(crs, algorithm)`
(spheroidal). Both store WKB values and are primitive types; their parameters
are metadata:
- **CRS** (both): a string, default `"OGC:CRS84"` (e.g. `srid:3857`,
`EPSG:4326`).
- **Edge algorithm** (geography only): one of `spherical` (default),
`vincenty`, `thomas`, `andoyer`, `karney`.
Gravitino is a catalog: we don't implement geospatial behavior — engines do.
Our job is to describe the column faithfully. Today these load as
`external(GEOMETRY)`, which drops the CRS/algorithm to a bare name and is
opaque to consumers (external types have caused downstream issues — Trino
#10957, DDL #11805), and the write path throws. Native types preserve the
CRS/algorithm round-trip and make the columns writable through Gravitino.
### Describe the solution
Two parameterized primitives — `Type(a)` and `Type(a, b)` where `Type
extends Type.PrimitiveType`:
```java
enum Name { ...; GEOMETRY; GEOGRAPHY; }
class GeometryType extends Type.PrimitiveType { String crs; }
// geometry(crs)
class GeographyType extends Type.PrimitiveType { String crs, algorithm; }
// geography(crs, algorithm)
```
- `simpleString()`: bare `geometry` / `geography` at defaults, else
`geometry(<crs>)` / `geography(<crs>,<algorithm>)`.
- **Metadata validation**: CRS is open per the spec (accept any non-empty
string); the edge algorithm has a fixed set, so validate it against the five
spec values at the type boundary. Both modeled as `String` in the
engine-neutral `api`.
- **`common` JsonUtils**: register the bare defaults + parameterized-form
regexes. One real detail: the parser lowercases the type string, but CRS is
case-sensitive (`OGC:CRS84`) — match against the original string to avoid
corrupting it.
- **`catalog-lakehouse-iceberg`**: `FromIcebergType`/`ToIcebergType`
branches, round-tripping CRS + algorithm (both ride the existing `atomic()`
path — no visitor hook, unlike `unknown`).
- **`clients/client-python`**, **docs**, **OpenAPI** `datatype.yaml`: mirror
the two types.
**Delivery:** geometry and geography share one design but ship as **two
stacked PRs** (geometry first, geography second) to keep each review small;
combined here only because the design is shared.
### Example usages
- A `geometry(OGC:CRS84)` column (store locations) loads as `geometry`
instead of `external(GEOMETRY)`, preserving the CRS.
- A `geography(OGC:CRS84, spherical)` column (flight paths) round-trips both
CRS and edge algorithm; and such columns become creatable through Gravitino at
`format-version = 3` (today the write path throws).
### Additional context
Series: `variant` (#11927 / #11932 / #11949), `unknown` (#11951). WKB values
are opaque and portable, so this is purely about how richly the catalog
describes the column — not data representability.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
