4ktLuffy commented on issue #23410: URL: https://github.com/apache/datafusion/issues/23410#issuecomment-5239310787
I can't reproduce this on `main` @ `308e212` — both DataFusion and PostgreSQL 14 return an empty result for the query as written. I checked four ways: 1. **The exact reproduction above** (in-memory tables) — empty. 2. **Ten variants around it** — filter placed before vs. after the subquery, a literal `a NOT IN (1, NULL)` list, a subquery mixing NULL and non-NULL values, a correlated subquery, and `(a != 1 OR a IS NULL)`. All ten match PostgreSQL exactly. 3. **Execution settings** — `target_partitions` at 1, 8 and 32, plus `filter_null_join_keys = true` and `repartition_joins = false`. All empty. (I checked partition counts because #5082 showed `pg_compat_window.slt` behaving differently at `target_partitions >= 32`.) 4. **CSV-backed external tables**, 200 rows, with the fixture asserted to contain exactly one NULL row — 0 rows. The plan does still push the filter beneath the null-aware anti join, which I believe is the shape flagged in the report: ``` logical_plan 01)Projection: t.id 02)--LeftAnti Join: t.a = __correlated_sq_1.k null_aware 03)----Filter: t.a != Int32(1) 04)------TableScan: t projection=[id, a] 05)----SubqueryAlias: __correlated_sq_1 06)------TableScan: u projection=[k] ``` I couldn't find a PR referencing this issue, so I don't know what changed in between. Is there context missing from the reproduction — a specific version, or a table provider other than in-memory? Happy to re-test if so. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
