Fabian Hueske created FLINK-18988:
-------------------------------------
Summary: Continuous query with LATERAL and LIMIT produces wrong
result
Key: FLINK-18988
URL: https://issues.apache.org/jira/browse/FLINK-18988
Project: Flink
Issue Type: Bug
Components: Table SQL / Planner
Affects Versions: 1.11.1
Reporter: Fabian Hueske
I was trying out the example queries provided in this blog post:
[https://materialize.io/lateral-joins-and-demand-driven-queries/] to check if
Flink supports the same and found that the queries were translated and executed
but produced the wrong result.
I used the SQL Client and Kafka (running at kafka:9092) to store the table
data. I executed the following statements:
{code:java}
-- create cities table
CREATE TABLE cities (
name STRING NOT NULL,
state STRING NOT NULL,
pop INT NOT NULL
) WITH (
'connector' = 'kafka',
'topic' = 'cities',
'properties.bootstrap.servers' = 'kafka:9092',
'properties.group.id' = 'mygroup',
'scan.startup.mode' = 'earliest-offset',
'format' = 'json'
);
-- fill cities table
INSERT INTO cities VALUES
('Los_Angeles', 'CA', 3979576),
('Phoenix', 'AZ', 1680992),
('Houston', 'TX', 2320268),
('San_Diego', 'CA', 1423851),
('San_Francisco', 'CA', 881549),
('New_York', 'NY', 8336817),
('Dallas', 'TX', 1343573),
('San_Antonio', 'TX', 1547253),
('San_Jose', 'CA', 1021795),
('Chicago', 'IL', 2695598),
('Austin', 'TX', 978908);
-- execute query
SELECT state, name
FROM
(SELECT DISTINCT state FROM cities) states,
LATERAL (
SELECT name, pop
FROM cities
WHERE state = states.state
ORDER BY pop
DESC LIMIT 3
);
-- result
state name
CA Los_Angeles
NY New_York
IL Chicago
-- expected result
state | name
------+-------------
TX | Dallas
AZ | Phoenix
IL | Chicago
TX | Houston
CA | San_Jose
NY | New_York
CA | San_Diego
CA | Los_Angeles
TX | San_Antonio
{code}
As you can see from the query result, Flink computes the top3 cities over all
states, not for every state individually. Hence, I assume that this is a bug in
the query optimizer or one of the rewriting rules.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
