I looked, but I am not much more enlightened. Maybe GEOMETRY/GEOGRAPHY will work a little like specialized forms VARIANT, which is really dynamically-typed.
Mihai ________________________________ From: David Chaava via dev <[email protected]> Sent: Thursday, November 5, 2026 3:45 AM To: [email protected] <[email protected]> Cc: David Chaava <[email protected]> Subject: Re: [DISCUSS] Planner-level GEOGRAPHY type in Apache Calcite Hi Mihai, Thanks for raising this. We have a design proposal available here: https://docs.google.com/document/d/1Btqd0oZT8ctqIaGI_AROyAFJQe68RsVyeQbtmK4wHQw/edit We would appreciate your feedback on it before implementation. The proposal introduces GEOGRAPHY as a statically known SQL and relational type. A column would be declared as GEOGRAPHY by its schema or adapter, and Calcite would preserve that type through validation and planning. Point, line, and polygon values would all have the same SQL type in this initial design. The runtime dispatch mentioned earlier was a possibility for spatial functions explicitly supporting both GEOMETRY and GEOGRAPHY. It would still require validation of the declared argument types. The first contribution focuses on the type itself and its propagation through a plan. The proposal includes assignment and coercion rules, nullability, nested types, and the intended behavior of comparisons and set operations. Equality and grouping are among the areas where we are seeking feedback on the guarantees Calcite needs. Could you help us understand which aspects you have in mind when referring to dynamic typing? We would appreciate your guidance on where the proposed typing rules need further clarification. Best, David On Wed, Nov 4, 2026 at 8:30 PM Mihai Budiu <[email protected]> wrote: > The typing rules for such types seem to be rather different from the rest > of Calcite — if I understand correctly, these constructs are generally > dynamically typed. So, I would be interested to see a design too prior to > the implementation. > > Mihai > ------------------------------ > *From:* David Chaava via dev <[email protected]> > *Sent:* Wednesday, November 4, 2026 6:19 AM > *To:* [email protected] <[email protected]> > *Cc:* David Chaava <[email protected]> > *Subject:* Re: [DISCUSS] Planner-level GEOGRAPHY type in Apache Calcite > > Hi Julian, Jerome, and Bertil, > > Thanks for the feedback and for sharing the implementation experience. > > Our immediate use case is preserving GEOGRAPHY through SQL validation and > relational planning in Calcite-based engines such as Flink. A query may > simply project an Iceberg geography column, so retaining its type is useful > even when no spatial functions are involved. > > Under the proposal, GEOGRAPHY would be a separate SQL type family. It would > not be a subtype of GEOMETRY in the argument-compatibility sense Julian > described. An existing function declared to accept GEOMETRY would reject a > GEOGRAPHY argument during validation. This prevents a geography value from > reaching an implementation with planar semantics accidentally. > > For spatial functions, support would need to be considered individually, > since some operations may share an implementation while others require > geodetic algorithms. Runtime dispatch could be an option for a function > explicitly declared to accept both types, provided its runtime > representation preserves the distinction. The validator would still need to > check that the supplied argument types are supported. > > As Jerome points out, the distinction can also matter at planning time, > when an adapter decides whether and how to push down an operation or an > engine chooses an index. The first contribution would make the type > distinction available to the planner. Pushdown rules and index selection > are outside its scope. > > Our proposed contribution is independently useful for queries that carry > GEOGRAPHY columns through a plan, with execution handled by the consuming > engine. It does not depend on adding spatial functions to Calcite. Any such > functions would need their own discussion of semantics and implementation > scope. > > On the SRID constraint, Bertil’s example explains our concern. A JTS > geometry with SRID 4326 still uses planar edges and operations. > Consequently, neither requiring an SRID nor fixing it to 4326 establishes > geodetic semantics. A richer constrained-type model might express that > distinction, and we understand the motivation to reduce duplicated > type-system code. For this contribution, we propose using Calcite’s > existing type machinery to express the distinction through a separate type > family. > > In the first contribution, Calcite’s GEOGRAPHY type would not encode CRS or > SRID in RelDataType, just as the existing GEOMETRY type does not encode > SRID there. How Flink handles CRS/SRID, edge semantics, and encoding > remains open in the FLIP discussion, where input from Sedona contributors > would be welcome. > > The SQL type distinction is also compatible with sharing runtime > components. Bertil’s suggestion to reuse JTS for representation, WKT/WKB > handling, and suitable accessors illustrates how code could be shared. > Jerome’s experience also helps identify the validation requirements. The > first contribution would leave runtime representation and serialization > unspecified. That keeps open the integration paths Jia outlined, including > RAW with Sedona-maintained adapters and a shared spatial type and SerDe. > > Regarding code size, we have a component breakdown rather than a reliable > line-count estimate at this stage. The intended changes are concentrated in > the following areas. > > - SQL syntax and conformance support for the type name, parsing, and > unparsing. > - SqlTypeName, SqlTypeFamily, assignment and coercion rules, and the > relevant type utilities and SQL/Rel conversion paths. > - Regression tests and documentation for nullability, nested types, type > propagation, mixed-type rejection, non-orderability, and SQL/Rel round > trips. > > We expect to reuse the existing nullability and type-composition machinery. > The scope excludes spatial function implementations, a new Java value > class, spatial library dependencies, and a general constrained-type > framework. CALCITE-1466 (unsigned types) and CALCITE-4918 (VARIANT) are > useful precedents for discussing the scope of type-system changes. A > preliminary patch would help identify the changes needed in existing > utilities and give us a concrete estimate of the size of the diff. > > Would this parser, validator, and relational type scope be acceptable for > the first contribution? We would also appreciate feedback on the equality > boundary already described in the proposal. It permits same-type equality, > with execution semantics and hashing supplied by the consuming engine or > adapter. Are additional planner-level guarantees needed for equality-based > operations such as DISTINCT and GROUP BY, particularly when those > operations may execute locally? > > Best, > David > > On Wed, Nov 4, 2026 at 11:14 AM Jerome Haltom <[email protected]> > wrote: > > > In my case, I am primarily trying to decide whether or not I can push an > > operation down or how to push the operation down. > > > > For an actual engine, knowing it at plan time will matter for index > > selection, etc. > > > > > > > > Get Outlook for Android<https://aka.ms/AAb9ysg> > > ________________________________ > > From: Julian Hyde <[email protected]> > > Sent: Tuesday, 03 November 2026 13:05:16 > > To: [email protected] <[email protected]> > > Subject: Re: [DISCUSS] Planner-level GEOGRAPHY type in Apache Calcite > > > > Is there actually a requirement to overload? The function could check at > > runtime whether it has a GEOMETRY or GEOGRAPHY value. > > > > > On Nov 3, 2026, at 10:54 AM, Jerome Haltom <[email protected]> > > wrote: > > > > > > It has been a couple months since I had thought through this, but I > > think the downside of my approach was there was no way to overload > > functions in a way that made sense. I don't think you could overload by > > constraint. > > > > > > So, so you couldn't validate. Which most engines do. > > > > > > ________________________________ > > > From: Julian Hyde <[email protected]> > > > Sent: Tuesday, November 03, 2026 12:11 > > > To: [email protected] <[email protected]> > > > Subject: Re: [DISCUSS] Planner-level GEOGRAPHY type in Apache Calcite > > > > > > There are a lot of geospatial functions. Would the functions that > > currently take GEOMETRY also be applicable to GEOGRAPHY? Do we need > > additional functions for the GEOGRAPHY type? > > > > > > Is GEOGRAPHY a subtype if GEOMETRY (in the sense that a GEOGRAPHY value > > can be used as an argument to a function with a GEOMETRY parameter? > > > > > > I worry about this change creating a lot of new code (possibly > > copy-pasted from existing code, possibly generated by agents, but > > nevertheless requiring maintenance). Give me a sense of the amount of new > > code that would be required, and which components it would reside in. > > > > > > I wonder whether GEOGRAPHY could be a GEOMETRY type with a constraint > > “srid IS NOT NULL”. We cannot currently attach constraints to types > (except > > the constraints that are represented as type parameters, e.g. VARCHAR(10) > > is “a string whose length is not greater than 10”). Is a constrained > > GEOMETRY type the right way to think of this? If so, attaching > constraints > > to types might be a way to reduce the amount of code that has to be > written. > > > > > > (I had a similar thought about constraints and types when we had the > > discussion about the VECTOR type. I still believe that the VECTOR type is > > basically an ARRAY type where the length is a constant. My work on Morel, > > and its elegant type system based on Standard ML’s modules and functors, > > makes me think about how we remove/reduce code by improving our type > > system.) > > > > > > Julian > > > > > > > > >> On Nov 3, 2026, at 8:24 AM, Jerome Haltom <[email protected]> > > wrote: > > >> > > >> I approve of this, as it would let me deprecate my existing > > geography-like support. > > >> > > >> > > > https://github.com/ikvmnet/calcite-dotnet/tree/main/src/Apache.Calcite.Geography > > >> > > >> This will be weird for you guys, since it's .NET code. But it's an > > attempt at a full suite of Geography functions across the existing > geometry > > types. It is only WGS84 however. It uses S2 and > > net.sf.geographiclib:GeographicLib-Java > > >> > > >> > > >> > > >> ________________________________ > > >> From: David Chaava via dev <[email protected]> > > >> Sent: Tuesday, November 03, 2026 08:51 > > >> To: [email protected] <[email protected]> > > >> Cc: [email protected] <[email protected]>; [email protected] < > > [email protected]>; David Chaava <[email protected]> > > >> Subject: [DISCUSS] Planner-level GEOGRAPHY type in Apache Calcite > > >> > > >> Hi Calcite community, > > >> > > >> We would like to discuss adding GEOGRAPHY as a first-class SQL and > > >> relational type in Calcite, distinct from the existing GEOMETRY type. > > >> > > >> This follows the discussion about native GEOGRAPHY support in Flink > [1], > > >> motivated in part by interoperability with Iceberg geography columns. > > >> Participants raised concerns about maintaining a Flink-only planner > > type, > > >> handling validation and type-system edge cases, and eventually > > migrating to > > >> an upstream Calcite type. We would like to address these concerns > > through a > > >> Calcite contribution. > > >> > > >> Our proposal is available here [2]. > > >> > > >> The proposed first change covers parsing, validation, relational type > > >> propagation, and unparsing, including nullability, type coercion > rules, > > >> nested types, and SQL/Rel round trips. A separate type family would > > >> preserve the distinction between geographic and planar values and > > prevent > > >> existing GEOMETRY functions from accepting GEOGRAPHY operands > > >> unintentionally. > > >> > > >> The initial type would be unparameterized and independent of a > > particular > > >> runtime representation. The proposal leaves CRS/SRID metadata, > encoding, > > >> and runtime execution to consuming engines and adapters. It does not > > >> introduce spatial functions, constructors, > > serialization/deserialization, > > >> or storage-format mappings. > > >> > > >> We also want to build on the Sedona discussion. Jia clarified that > > >> SedonaFlink already supports Geography through RAW with a dedicated > > >> serializer, and called for joint consideration of CRS/SRID, > > spherical-edge > > >> semantics, validation, equality, and shared serialization. The limited > > >> Calcite scope proposed here is intended as a first step; it does not > > settle > > >> those broader interoperability choices or assume that Sedona would > > adopt a > > >> particular runtime representation. > > >> > > >> We would appreciate feedback on two main questions: > > >> > > >> 1. Should Calcite expose an unparameterized, representation-neutral > > >> GEOGRAPHY type distinct from GEOMETRY? > > >> 2. Is the proposed parser/validator/type-system scope sufficient for > a > > >> first contribution without constructors or a Calcite runtime value > > class? > > >> > > >> In considering these questions, we would particularly welcome feedback > > on > > >> the proposed semantic boundary: same-type equality is permitted with > > >> adapter-defined execution semantics, ordering is rejected, and > CRS/SRID > > is > > >> not encoded in the Calcite type. Are additional guarantees needed at > the > > >> planner level? > > >> > > >> Subject to Calcite review and acceptance, the intended integration > path > > is > > >> to merge the necessary support upstream, then bring those changes into > > >> Flink, with any temporary copies removed when Flink upgrades its > Calcite > > >> dependency. We would also welcome suggestions for regression cases, > > >> particularly around null handling, nested types, and type-to-SQL > > conversion. > > >> > > >> The Flink dev list and Jia are copied to help coordinate the > discussion. > > >> Thank you for your feedback. > > >> > > >> Best regards, > > >> David > > >> > > >> [1] Flink GEOGRAPHY discussion > > >> <https://lists.apache.org/thread/flbj8dfdfgs26klrxt7xch3r9785ky67> > > >> [2] Planner-level GEOGRAPHY type in Apache Calcite proposal > > >> < > > > https://docs.google.com/document/d/1Btqd0oZT8ctqIaGI_AROyAFJQe68RsVyeQbtmK4wHQw/edit > > > > > > > > > > >
