devanbenz commented on issue #26169: URL: https://github.com/apache/datafusion/issues/26169#issuecomment-6722961404
> I have set up 1TB TPCH sandbox for latest datafusion on `main` (`a616831`) > > - DataFusion `57.0.0-dev`, `main` at [`a616831`](https://github.com/apache/datafusion/commit/a616831122aa525ed843ee3c5e11f4803f557c98) (2026-11-06) > - Hardware: k8s pod, amd64, 32 cores > - Data: TPC-H SF1000 Parquet, zstd, 512 MB blocks, partitioned, read from S3 > - `target_partitions = 32` > - `datafusion.runtime.memory_limit = 105G` (greedy pool) > - `datafusion.runtime.temp_directory = /data`, `max_temp_directory_size = 900G` > - `datafusion.runtime.metadata_cache_limit = 2G`, `list_files_cache_limit = 256M`, `file_statistics_cache_limit = 256M` > - 3 iterations per query. The times below are in seconds. > > **Results** > > | Query | Mean | Min | Max | Status | > |---|---:|---:|---:|---| > | Q01 | 46.17 | 45.86 | 46.38 | ok | > | Q02 | 37.71 | 35.83 | 41.20 | ok | > | Q03 | 31.25 | 29.53 | 34.22 | ok | > | Q04 | 22.92 | 22.58 | 23.18 | ok | > | Q05 | 53.85 | 47.61 | 64.85 | ok | > | Q06 | 4.76 | 4.63 | 4.84 | ok | > | Q07 | 61.13 | 59.27 | 62.28 | ok | > | Q08 | 54.80 | 53.02 | 56.15 | ok | > | Q09 | | | | **failed (OOM)** | > | Q10 | 37.20 | 36.27 | 38.02 | ok | > | Q11 | 13.44 | 12.13 | 14.64 | ok | > | Q12 | 14.86 | 13.94 | 15.80 | ok | > | Q13 | 29.77 | 29.43 | 30.32 | ok | > | Q14 | 4.01 | 3.50 | 4.57 | ok | > | Q15 | 16.55 | 10.93 | 22.36 | ok | > | Q16 | 11.52 | 10.22 | 13.70 | ok | > | Q17 | 75.21 | 74.55 | 76.05 | ok | > | Q18 | | | | **failed (OOM)** | > | Q19 | 31.41 | 30.92 | 31.77 | ok | > | Q20 | 24.35 | 23.23 | 25.99 | ok | > | Q21 | | | | **failed (OOM)** | > | Q22 | 22.63 | 20.17 | 24.50 | ok | > > 19 of 22 queries passed. The sum of their means is 593.6 s and the geometric mean is 24.4 s. > > **Failures** > > Q09, Q18 and Q21 failed on all 3 iterations with `Resources exhausted` while building hash join input. `HashJoinInput` reports `can spill: false`, so the 105G pool fills up with build-side reservations of similar size, one per partition. > > - **Q09**: `Failed to allocate additional 763.0 MB for HashJoinInput[24] with 763.0 MB already allocated for this reservation - 361.1 MB remain available for the total memory pool: greedy(used: 104.6 GB, pool_size: 105.0 GB)`. The top consumers are `HashJoinInput` partitions at 2.1 GB each. > - **Q18**: `Failed to allocate additional 1440.0 KB for HashJoinInput[4] with 2.5 GB already allocated for this reservation - 247.2 KB remain available for the total memory pool: greedy(used: 105.0 GB, pool_size: 105.0 GB)`. The top consumers are `HashJoinInput` partitions at 2.5 GB each. > - **Q21**: `Failed to allocate additional 4.7 GB for HashJoinInput[24] with 0.0 B already allocated for this reservation - 1591.6 MB remain available for the total memory pool: greedy(used: 103.4 GB, pool_size: 105.0 GB)`. The top consumers are `HashJoinInput` partitions at 6.5 GB each. > > > I'll run polars soon, so we can track weak DF parts and define next steps Is this with or without filter pushdown? Might be good to do a run with filter pushdown on as well if not 🤔 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
