Yibo Dong created SPARK-61972:
---------------------------------
Summary: UNION ALL of different CHAR lengths changes filtered rows
when branches are reversed
Key: SPARK-61972
URL: https://issues.apache.org/jira/browse/SPARK-61972
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 5.0.0
Environment: Spark 5.0.0-SNAPSHOT, upstream master at
\{{dada2a6db5d73b152d6ba16e4fe21406f86abfe3}} (2026-11-15), local mode, default
SQL settings
Reporter: Yibo Dong
h2. What happened
A filter above {{UNION ALL}} drops a matching CHAR row, and reversing the
branches changes which row is returned.
h2. How to reproduce
Run in a fresh Spark SQL session with a fresh warehouse. The two final queries
differ only in branch order.
{code:sql}
CREATE TABLE t3 (id INT, c CHAR(3)) USING PARQUET;
CREATE TABLE t5 (id INT, c CHAR(5)) USING PARQUET;
INSERT INTO t3 VALUES (1, 'A');
INSERT INTO t5 VALUES (2, 'A');
SELECT id
FROM (
SELECT id, c FROM t3
UNION ALL
SELECT id, c FROM t5
) u
WHERE c = 'A'
ORDER BY id;
SELECT id
FROM (
SELECT id, c FROM t5
UNION ALL
SELECT id, c FROM t3
) u
WHERE c = 'A'
ORDER BY id;
{code}
h2. Expected result
Both queries should return the same rows regardless of branch order. Under the
default CHAR padding semantics, both rows should match the short literal,
returning IDs 1 and 2.
h2. Actual result
The CHAR(3)-first query returns only ID 1, while the CHAR(5)-first query
returns only ID 2.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]