adriangb opened a new pull request, #27090: URL: https://github.com/apache/datafusion/pull/27090
## Which issue does this PR close? - This PR does not close an issue. It adds a baseline for a follow-up PR that changes how a `Partitioned` hash join with dense integer keys filters its probe side. ## Rationale for this change No `hj` query runs a `Partitioned` join on integer keys whose build partitions are too large for an IN list. Such a join builds an `ArrayMap` per partition when the keys are dense, and a hash table when they are sparse. The suite has no query of either kind, so a change to how these joins filter the probe side has nothing to compare against. ## What changes are included in this PR? One commit. It adds two `hj` queries over TPC-H `orders`, joined to itself on `o_custkey` in `Partitioned` mode: | Query | Build side | Keys | |---|---|---| | Q26 | orders of the first 40K customers (~270K rows) | dense, about 10 rows per key | | Q27 | orders of customers whose key is a multiple of 37 | sparse | Both probe 15M rows with about a 2.7% hit rate. The runner now also registers `orders`. The queries set no Parquet options, so a caller can run them with or without filter pushdown, for example with `DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true`. ## What is the testing strategy for this PR? The change adds benchmark queries only. I ran both queries on TPC-H SF10 (16 partitions, 10 iterations) and checked their plans: both use `HashJoinExec: mode=Partitioned`. With pushdown on, Q26's probe scan gets a per-partition routed filter over `ArrayMap` partitions, and Q27's gets a merged hash lookup. | Query | `pushdown_filters=false` | `pushdown_filters=true` | |---|---|---| | Q26 (dense) | 50.0 ms | 93.5 ms | | Q27 (sparse) | 54.5 ms | 59.2 ms | ## Are there any user-facing changes? No. 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
