lyne7-sc opened a new issue, #2390: URL: https://github.com/apache/auron/issues/2390
**Describe the bug** <!-- A clear and concise description of what the bug is. --> Native Iceberg scans do not preserve Spark's key-grouped input partitions. This can cause incorrect results in storage-partitioned joins. **To Reproduce** <!-- Steps to reproduce the behavior: 1. Go to '...' 2. Click on '....' 3. Scroll down to '....' 4. See error --> Create two Iceberg tables partitioned by the join key: ```sql CREATE TABLE local.db.t_left (id INT, p INT) USING iceberg PARTITIONED BY (p); INSERT INTO local.db.t_left VALUES (0, 0), (1, 1); CREATE TABLE local.db.t_right (value INT, p INT) USING iceberg PARTITIONED BY (p); INSERT INTO local.db.t_right VALUES (10, 0), (11, 0), (12, 1); ``` Enable storage-partitioned joins: spark.sql.sources.v2.bucketing.enabled=true spark.sql.iceberg.planning.preserve-data-grouping=true Run a sort-merge join: ```sql SELECT /*+ MERGE(l, r) */ l.id, l.p, r.value FROM local.db.t_left l JOIN local.db.t_right r ON l.p = r.p; ``` **Expected behavior** <!-- A clear and concise description of what you expected to happen. --> The query returns: (0, 0, 10) (0, 0, 11) (1, 1, 12) **Actual result** The query returns only one row: (0, 0, 11) The expected rows (0, 0, 10) and (1, 1, 12) are missing. **Screenshots** <!-- If applicable, add screenshots to help explain your problem. --> **Additional context** <!-- Add any other context about the problem here. --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
