[
https://issues.apache.org/jira/browse/FLINK-40496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arvid Heise resolved FLINK-40496.
---------------------------------
Fix Version/s: 2.4.0
Resolution: Fixed
Merged into master as 7d194385601701283df5da9f296f3eb5c61b46ba.
> CREATE OR ALTER MATERIALIZED TABLE applies column-change rules inconsistently
> with ALTER ... AS and can silently reorder existing columns
> -----------------------------------------------------------------------------------------------------------------------------------------
>
> Key: FLINK-40496
> URL: https://issues.apache.org/jira/browse/FLINK-40496
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Planner
> Affects Versions: 2.3.0
> Reporter: Arvid Heise
> Assignee: Arvid Heise
> Priority: Major
> Labels: pull-request-available
> Fix For: 2.4.0
>
>
> h2. What happens
> CREATE OR ALTER derives a materialized table's column changes differently
> from ALTER ... AS and does not surface a query-driven column reorder to the
> append-only validation. Reordering existing columns via CREATE OR ALTER is
> therefore handled inconsistently:
> * when the query text is unchanged (e.g. only a bare column list reorders the
> columns), the reorder is silently applied - the stored column order is
> rewritten with no error;
> * when the query text changes, the same reorder is rejected.
> The silent path leaves the stored schema in an order that no longer matches
> the query. The refresh runs a positional INSERT INTO <table> <expandedQuery>
> with no explicit column list, so the mismatch then miscompiles with a sink
> type error, e.g. "Incompatible types for sink column ... at position ...".
> Example - existing materialized table users_shops = (user_id, shop_id, ds,
> order_cnt):
> {code:sql}
> CREATE OR ALTER MATERIALIZED TABLE users_shops (shop_id, user_id, ds,
> order_cnt)
> AS SELECT user_id, shop_id, ds, COUNT(order_id) AS order_cnt FROM ...
> {code}
> The bare list reorders user_id/shop_id while the query is unchanged, so no
> ModifyDefinitionQuery is produced and the append-only check is skipped: the
> columns are silently reordered to (shop_id, user_id, ds, order_cnt).
> h2. Fix
> Derive the CREATE OR ALTER column diff from the query the same way ALTER ...
> AS does, and apply the append-only column rules to every query-carrying alter
> (CREATE OR ALTER and ALTER ... AS) regardless of whether the query text
> changed. Reordering or retyping existing columns is then rejected
> consistently across both statements.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
