Xingcan Cui created FLINK-36004:
-----------------------------------
Summary: Flink SQL returns wrong results for Paimon tables with
complex schemas
Key: FLINK-36004
URL: https://issues.apache.org/jira/browse/FLINK-36004
Project: Flink
Issue Type: Bug
Components: Table SQL / Runtime
Affects Versions: 1.19.1, 1.18.1
Reporter: Xingcan Cui
We have a Paimon table with some nested files such as the following one.
{code:java}
`f1` ROW <
`f2` ROW <
`f3` ARRAY < ROW < `key` INT, `value` FLOAT > NOT NULL >,
`f4` ARRAY < ROW < `key` INT, `value` STRING > NOT NULL >,
`f5` ARRAY < ROW < `key` BIGINT, `value` FLOAT > NOT NULL >,
`f6` ARRAY < ROW < `key` BIGINT, `value` BIGINT > NOT NULL >,
`f7` ARRAY < ROW < `key` BIGINT, `value` ROW < `f71` ARRAY <
FLOAT > > > NOT NULL >,
`f8` ARRAY < ROW < `f81` STRING, `f82` STRING, `f83` BIGINT >
NOT NULL >,
`f9` ARRAY < ROW < `key` BIGINT, `value` ROW < `f91` ARRAY <
BIGINT > > > NOT NULL >,
`f10` ROW < `f101` ARRAY < ROW < `f102` BIGINT, `f103` STRING,
`f104` BIGINT > NOT NULL > >,
`f11` ARRAY < ROW < `key` BIGINT, `value` STRING > NOT NULL >
>
{code}
When a select query includes some nested columns, the results will be wrong.
For example, {{SELECT CARDINALITY(f1.f2.f3) AS r FROM...WHERE...}} can return
correct results but {{SELECT CARDINALITY(f1.f2.f3) AS r, f1 FROM...WHERE...}}
will return wrong values for {{r.}}
The query execution won't throw any exception but fails silently.
I'm not sure if this is a Paimon-specific issue, but I also tested running the
same query with Spark and StarRocks, and both of them can produce correct
results.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
