[
https://issues.apache.org/jira/browse/FLINK-41279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ramin Gharib updated FLINK-41279:
---------------------------------
Release Note: Flink SQL accepts the infix cast operator expr::type as a
shorthand for CAST(expr AS type), for example payload['id']::BIGINT. It binds
tighter than arithmetic and comparison operators, so a::INT + 1 means CAST(a AS
INT) + 1. A '[' directly after the target type is rejected. Write
(v::VARIANT)['id'] to access a field or element of the result.
Description:
Flink SQL only supports {{CAST(x AS type)}}. PostgreSQL, DuckDB and Snowflake
also accept the shorter infix form {{x::type}}, and Calcite supports it in its
Babel parser. These statements fail to parse in Flink today:
{code:sql}
SELECT '42'::INT + 1; -- CAST('42' AS INT) + 1
SELECT payload::VARIANT FROM t; -- CAST(payload AS VARIANT)
SELECT m['k']::INT FROM t; -- CAST(m['k'] AS INT)
SELECT s::INT::STRING FROM t; -- CAST(CAST(s AS INT) AS STRING)
{code}
h3. Proposal
Treat {{::}} as pure syntax for {{CAST}}. The parser turns {{x::type}} into a
plain {{CAST(x AS type)}} call, so validation, planning, code generation and
compiled plans stay unchanged.
* Add the {{::}} token and an {{InfixCast}} production through the existing
{{binaryOperatorsTokens}} and {{extraBinaryExpressions}} hooks in
{{Parser.tdd}} and {{parserImpls.ftl}}. {{Parser.jj}} stays untouched.
* The production emits a Flink-side binary operator whose {{createCall}}
returns a {{CAST}} call. It keeps Calcite's precedence for {{::}}: tighter than
arithmetic, looser than {{[]}} and {{.}}.
* The type after {{::}} accepts what {{CAST}} accepts: Flink's {{DataType()}}
production, so types like {{ARRAY<INT>}}, {{ROW<...>}} and {{VARIANT}} work,
plus {{INTERVAL}} qualifiers.
h3. Why not reuse Calcite's SqlLibraryOperators.INFIX_CAST
Calcite's operator stays in the validated tree and unparses as {{a :: INT}}.
That text ends up in persisted catalog state, such as the expanded queries of
views and materialized tables and computed columns, and older Flink versions
can't parse it. The unparse format also belongs to Calcite: CALCITE-7475
changed it in Calcite 1.42. Desugaring in the parser keeps that unparsed text
as {{CAST(a AS INT)}} and leaves no new operator for later Calcite upgrades to
track. The original query of a view or materialized table is stored as written,
so it keeps {{::}} under either design.
h3. Out of scope
* Table API. {{BaseExpressions#cast}} already covers it.
* A {{TRY_CAST}} counterpart. PostgreSQL has none.
* Postfix access directly after the type, like {{x::ARRAY<INT>[1]}}, which
Calcite Babel added in CALCITE-7475.
was:
Flink SQL only supports {{CAST(x AS type)}}. PostgreSQL, DuckDB and Snowflake
also accept the shorter infix form {{x::type}}, and Calcite supports it in its
Babel parser. These statements fail to parse in Flink today:
{code:sql}
SELECT '42'::INT + 1; -- CAST('42' AS INT) + 1
SELECT payload::VARIANT FROM t; -- CAST(payload AS VARIANT)
SELECT m['k']::INT FROM t; -- CAST(m['k'] AS INT)
SELECT s::INT::STRING FROM t; -- CAST(CAST(s AS INT) AS STRING)
{code}
h3. Proposal
Treat {{::}} as pure syntax for {{CAST}}. The parser turns {{x::type}} into a
plain {{CAST(x AS type)}} call, so validation, planning, code generation and
compiled plans stay unchanged.
* Add the {{::}} token and an {{InfixCast}} production through the existing
{{binaryOperatorsTokens}} and {{extraBinaryExpressions}} hooks in
{{Parser.tdd}} and {{parserImpls.ftl}}. {{Parser.jj}} stays untouched.
* The production emits a Flink-side binary operator whose {{createCall}}
returns a {{CAST}} call. It keeps Calcite's precedence for {{::}}: tighter than
arithmetic, looser than {{[]}} and {{.}}.
* The type after {{::}} goes through Flink's {{DataType()}} production, so
types like {{ARRAY<INT>}}, {{ROW<...>}} and {{VARIANT}} work.
h3. Why not reuse Calcite's SqlLibraryOperators.INFIX_CAST
Calcite's operator stays in the validated tree and unparses as {{a :: INT}}.
That text ends up in persisted catalog state, such as view expanded queries,
computed columns and materialized table definition queries, and older Flink
versions can't parse it. The unparse format also belongs to Calcite:
CALCITE-7475 changed it in Calcite 1.42. Desugaring in the parser keeps all
stored text as {{CAST(a AS INT)}} and leaves no new operator for later Calcite
upgrades to track.
h3. Out of scope
* Table API. {{BaseExpressions#cast}} already covers it.
* A {{TRY_CAST}} counterpart. PostgreSQL has none.
* Postfix access directly after the type, like {{x::ARRAY<INT>[1]}}, which
Calcite Babel added in CALCITE-7475.
> Support the :: infix cast operator
> ----------------------------------
>
> Key: FLINK-41279
> URL: https://issues.apache.org/jira/browse/FLINK-41279
> Project: Flink
> Issue Type: New Feature
> Components: Table SQL / API
> Reporter: Ramin Gharib
> Assignee: Ramin Gharib
> Priority: Major
> Labels: pull-request-available
>
> Flink SQL only supports {{CAST(x AS type)}}. PostgreSQL, DuckDB and Snowflake
> also accept the shorter infix form {{x::type}}, and Calcite supports it in
> its Babel parser. These statements fail to parse in Flink today:
> {code:sql}
> SELECT '42'::INT + 1; -- CAST('42' AS INT) + 1
> SELECT payload::VARIANT FROM t; -- CAST(payload AS VARIANT)
> SELECT m['k']::INT FROM t; -- CAST(m['k'] AS INT)
> SELECT s::INT::STRING FROM t; -- CAST(CAST(s AS INT) AS STRING)
> {code}
> h3. Proposal
> Treat {{::}} as pure syntax for {{CAST}}. The parser turns {{x::type}} into a
> plain {{CAST(x AS type)}} call, so validation, planning, code generation and
> compiled plans stay unchanged.
> * Add the {{::}} token and an {{InfixCast}} production through the existing
> {{binaryOperatorsTokens}} and {{extraBinaryExpressions}} hooks in
> {{Parser.tdd}} and {{parserImpls.ftl}}. {{Parser.jj}} stays untouched.
> * The production emits a Flink-side binary operator whose {{createCall}}
> returns a {{CAST}} call. It keeps Calcite's precedence for {{::}}: tighter
> than arithmetic, looser than {{[]}} and {{.}}.
> * The type after {{::}} accepts what {{CAST}} accepts: Flink's {{DataType()}}
> production, so types like {{ARRAY<INT>}}, {{ROW<...>}} and {{VARIANT}} work,
> plus {{INTERVAL}} qualifiers.
> h3. Why not reuse Calcite's SqlLibraryOperators.INFIX_CAST
> Calcite's operator stays in the validated tree and unparses as {{a :: INT}}.
> That text ends up in persisted catalog state, such as the expanded queries of
> views and materialized tables and computed columns, and older Flink versions
> can't parse it. The unparse format also belongs to Calcite: CALCITE-7475
> changed it in Calcite 1.42. Desugaring in the parser keeps that unparsed text
> as {{CAST(a AS INT)}} and leaves no new operator for later Calcite upgrades
> to track. The original query of a view or materialized table is stored as
> written, so it keeps {{::}} under either design.
> h3. Out of scope
> * Table API. {{BaseExpressions#cast}} already covers it.
> * A {{TRY_CAST}} counterpart. PostgreSQL has none.
> * Postfix access directly after the type, like {{x::ARRAY<INT>[1]}}, which
> Calcite Babel added in CALCITE-7475.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
