[
https://issues.apache.org/jira/browse/FLINK-40267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated FLINK-40267:
-----------------------------------
Labels: pull-request-available (was: )
> CREATE MATERIALIZED TABLE skips persisted-metadata-column validation applied
> by CREATE OR ALTER
> -----------------------------------------------------------------------------------------------
>
> Key: FLINK-40267
> URL: https://issues.apache.org/jira/browse/FLINK-40267
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Planner
> Affects Versions: 2.2.0, 2.3.0, 2.4.0
> Reporter: Arvid Heise
> Assignee: Arvid Heise
> Priority: Major
> Labels: pull-request-available
> Fix For: 2.4.0
>
>
> SqlCreateOrAlterMaterializedTableConverter#getMergedSchema selects one of two
> validators for the same invariant — every persisted column in the schema part
> must be produced by the query part:
> - CREATE OR ALTER →
> MaterializedTableUtils#validatePersistedColumnsUsedByQuery: checks physical
> columns and non-virtual metadata columns, unwraps SqlTableColumnPosition.
> - plain CREATE → converter-private validatePhysicalColumnsUsedByQuery:
> continues on anything that is not a SqlRegularColumn.
> A CREATE MATERIALIZED TABLE declaring a persisted metadata column absent from
> the AS SELECT is therefore accepted. The materialized table is created, and
> the refresh pipeline fails at compile time in DynamicSinkUtils with "Column
> types of query result and sink do not match / Different number of columns",
> because persisted metadata columns are part of the sink's consumed row type.
>
> {noformat}
> CREATE TABLE src (order_id STRING NOT NULL, price DOUBLE);
> CREATE MATERIALIZED TABLE mt (
> `order_id` STRING NOT NULL,
> `price` DOUBLE,
> `kafka_ts` TIMESTAMP_LTZ(3) METADATA FROM 'timestamp'
> ) AS SELECT order_id, price FROM src;
> {noformat}
>
> The identical statement written as CREATE OR ALTER on a non-existing table is
> rejected synchronously with a precise message, which contradicts the FLIP-546
> equivalence of the two forms.
> The split originates in FLINK-38674 (physical-only check, added inline for
> schema definition on CREATE) and FLINK-38766 (persisted check, added for
> ALTER MATERIALIZED TABLE ADD). FLINK-39700 extracted the former and routed
> only CREATE OR ALTER to the latter.
> Fix: call validatePersistedColumnsUsedByQuery on both branches and drop
> validatePhysicalColumnsUsedByQuery (strict superset). Parameterize the
> statement kind in PERSISTED_COLUMN_NOT_USED_IN_QUERY, whose prefix is
> hardcoded to "Failed to execute ALTER MATERIALIZED TABLE statement." and
> already misfires on the CREATE OR ALTER path. Update the message assertions
> in SqlMaterializedTableNodeToOperationConverterTest and add CREATE-side
> coverage.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
