Ferenc Csaky created FLINK-33353:
------------------------------------
Summary: SQL fails because "TimestampType.kind" is not serialized
Key: FLINK-33353
URL: https://issues.apache.org/jira/browse/FLINK-33353
Project: Flink
Issue Type: Bug
Components: Table SQL / API
Affects Versions: 1.18.0
Reporter: Ferenc Csaky
We have a custom persistent catalog store, which stores tables, views etc. in a
DB. In our application, it is required to utilize the serialized formats of
entities, but the same applies to the Hive, as it functions as a persistent
catalog.
Take the following example SQL:
{code:sql}
CREATE TABLE IF NOT EXISTS `txn_gen` (
`txn_id` INT,
`amount` INT,
`ts` TIMESTAMP(3),
WATERMARK FOR `ts` AS `ts` - INTERVAL '1' SECOND
) WITH (
'connector' = 'datagen',
'fields.txn_id.min' = '1',
'fields.txn_id.max' = '5',
'rows-per-second' = '1'
);
CREATE VIEW IF NOT EXISTS aggr_ten_sec AS
SELECT txn_id,
TUMBLE_ROWTIME(`ts`, INTERVAL '10' SECOND) AS w_row_time,
COUNT(txn_id) AS txn_count
FROM txn_gen
GROUP BY txn_id, TUMBLE(`ts`, INTERVAL '10' SECOND);
SELECT txn_id,
SUM(txn_count),
TUMBLE_START(w_row_time, INTERVAL '20' SECOND) AS total_txn_count
FROM aggr_ten_sec
GROUP BY txn_id, TUMBLE(w_row_time, INTERVAL '20' SECOND);
{code}
This will work without any problems when we simply execute it in a
{{TableEnvironment}}, but it fails with the below error when we try to execute
the query based on the serialized table metadata.
{code}
org.apache.flink.table.api.TableException: Window aggregate can only be defined
over a time attribute column, but TIMESTAMP(3) encountered.
{code}
If there is a view which would require to use ROWTIME, it will be lost and we
cannot recreate the same query from the serialized entites.
Currently in {{TimestampType}} the "kind" field is deliberatly annotated as
{{@Internal}} and is not serialized, although it breaks this functionality.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
