Dandandan opened a new pull request, #24437: URL: https://github.com/apache/datafusion/pull/24437
## Which issue does this PR close? - N/A ## Rationale for this change For a join on more than one column the join key is the *tuple* of all key columns, but `estimate_inner_join_cardinality` used the distinct count of the single most selective column. A composite key is more selective than any of its parts, so the estimate comes out too large. TPC-H q9 joins `partsupp` on `(ps_partkey, ps_suppkey)` — its primary key, so 800k distinct pairs — while the largest per-column count is only 200k. At SF1 that join is estimated at 4,800,972 rows; the true answer is 319,404. With this change it estimates 800,000. ## What changes are included in this PR? Estimate the distinct count of the key tuple instead: it can be no larger than the product of the per-column counts, and no larger than the number of rows, so take the smaller of the two. Single-column joins are unaffected — the product of one count is that count, and `max_distinct_count` already caps it at the row count. ## Are these changes tested? Yes. `test_inner_join_cardinality_multiple_column` is updated for the new bound, and `test_inner_join_cardinality_multiple_column_below_row_count` is added to cover the case where the product rather than the row count is the binding limit. I also compared the build and probe row counts of every hash join across TPC-H SF1 (62 joins, from `dfbench tpch --debug` metrics) before and after: the results are identical, so this changes no TPC-H plan and carries no measured runtime effect either way. It is an estimate accuracy improvement, not a performance fix. ## Are there any user-facing changes? No API changes. Multi-column joins may pick a different build side, though none do in TPC-H. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
