LuciferYang opened a new pull request, #14658:
URL: https://github.com/apache/gravitino/pull/14658

   ### What changes were proposed in this pull request?
   
   - `BaseCatalog.createTable` throws `UnsupportedOperationException` when the 
table has a computed column, a metadata column or a watermark. The add-column 
path of `alterTable` does the same for computed and metadata columns; watermark 
changes in `ALTER TABLE` were already rejected.
   - In the Hive catalog, the error suggests creating the table as a Flink 
generic table, which keeps them in `flink.*` properties. Generic tables don't 
go through this path and are unchanged.
   - The Flink connector docs list the limitation.
   
   ### Why are the changes needed?
   
   Gravitino columns have no expression or metadata key, so these columns were 
stored as plain physical columns and the watermark was dropped, with no error. 
The table then had a different schema from the one declared. For `t (a INT, b 
AS a + 1)`, `INSERT INTO t VALUES (1)` fails with "Different number of 
columns", and after `INSERT INTO t (a) VALUES (1)` the value of `b` is NULL 
instead of 2.
   
   Fixes #14656
   
   ### Does this PR introduce _any_ user-facing change?
   
   Yes. `CREATE TABLE` and `ALTER TABLE ... ADD` with these now fail in the 
Paimon, Iceberg and JDBC catalogs and for Hive tables with 
`'connector'='hive'`. `CREATE TABLE IF NOT EXISTS` with them also fails when 
the table already exists. Paimon's own Flink catalog keeps these in table 
options, so the Gravitino Paimon catalog is now stricter than it; storing them 
the same way would be a separate change.
   
   ### How was this patch tested?
   
   - New `TestBaseCatalog` tests, run in each Flink version module: 
`createTable` rejects a computed column, a metadata column and a watermark, and 
still creates a table with only physical columns; `alterTable` rejects adding a 
computed column and still adds a physical one; the generic table advice appears 
only in catalogs that support generic tables.
   - Reproduced the bug on main with the Flink 1.20 integration test 
environment for the Paimon, Iceberg and MySQL catalogs.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to