Rich-T-kid commented on PR #12008: URL: https://github.com/apache/arrow-rs/pull/12008#issuecomment-6542922234
The controlled DataFusion end-to-end run is complete: [results](https://github.com/apache/datafusion/pull/26964#issuecomment-6539898972). Using DataFusion #26816 with every Arrow crate pinned to either `bf0cde345b14e5abdf3e8abb57f55d43da48b49f` (this PR's parent) or `90cbf31cd8dbacb5ae12566c5206624961461f90` (this PR), `clickbench_partitioned` with dictionary reads enabled changed from **50.86 s to 40.58 s (-20.2%)**. The queries expected to concatenate sliced dictionary group keys improved most: | Query | Parent | This PR | Change | | --- | ---: | ---: | ---: | | Q5 | 1035.92 ms | 826.53 ms | 1.25x | | Q12 | 1062.14 ms | 844.74 ms | 1.26x | | Q13 | 1551.81 ms | 1298.19 ms | 1.20x | | Q14 | 1160.91 ms | 913.46 ms | 1.27x | | Q16 | 2175.27 ms | 1379.22 ms | 1.58x | | Q17 | 2172.82 ms | 1386.31 ms | 1.57x | | Q18 | 3502.00 ms | 2033.89 ms | 1.72x | | Q33 | 6963.34 ms | 3857.07 ms | 1.81x | | Q34 | 7235.37 ms | 3948.00 ms | 1.83x | No spill occurred on either side. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
