Hi everyone,

I'd like to start a discussion on a FLIP that adds three SQL ergonomics
features to Flink SQL: GROUP BY <ordinal>, GROUP BY ALL, and ORDER BY ALL.

FLIP:
https://docs.google.com/document/d/16R83T86X1ATmmPe_QFnnUMdnpiOcvyMrUOOctmm59Gk/edit?usp=sharing

*Motivation*
Users migrating to Flink SQL from Snowflake, DuckDB, BigQuery, and others
expect positional grouping and ALL grouping/sorting. Their absence forces
query rewrites during migration and raises time-to-first-query. These are
validation-time conveniences that expand into standard grouping/sorting, so
the planning and execution pipeline is unchanged across streaming and batch.

*Summary*
- GROUP BY <ordinal>: "GROUP BY 1, 2" groups by the 1st and 2nd SELECT
expressions. This redefines today's "GROUP BY <constant>" behavior, so it
is a breaking change - gated behind a config option, default off. It reuses
Calcite's existing isGroupByOrdinal() resolution, no custom code.
- GROUP BY ALL: groups by every SELECT expression that is not an aggregate
or window function.
- ORDER BY ALL: sorts by every projected column, left to right, with an
optional trailing ASC/DESC and NULLS FIRST/NULLS LAST applied to all keys.

All three features are gated behind Boolean options in TableConfigOptions,
default false, for a staged rollout. With the options off, behavior is
unchanged.

GROUP BY ALL and ORDER BY ALL are net-new syntax and depend on Calcite
changes already merged upstream (CALCITE-7594
<https://github.com/apache/calcite/pull/5009>, CALCITE-7597
<https://github.com/apache/calcite/pull/5010>), targeted for Calcite
1.43.0; SELECT * support in both is in review (CALCITE-7647
<https://github.com/apache/calcite/pull/5089>). Because of this, the GROUP
BY ALL / ORDER BY ALL parts of this FLIP once it upgrades its Calcite
dependency to a release that includes these changes. The FLIP itself can be
discussed and accepted now; merging / releasing those two features is gated
by that Calcite upgrade. GROUP BY <ordinal> has no such dependency - it
reuses Calcite functionality that already exists.

Thanks,
Tisya Bhatia

Reply via email to