Rachelint commented on PR #11802: URL: https://github.com/apache/datafusion/pull/11802#issuecomment-2270976623
> > @alamb It mainly improve the short queries in `clickbench_partitioned`, and In my expectations, other cases should have no changes, maybe can try the `clickbench_partitioned` case? > > That is my expectation too > > > 🤔 But it is strange that it get slower in other cases (especially the not short queries in `clickbench_1`), I guess the reasons chould be: > > > > * Some changes in the latest `main` but not in this `branch` make a difference? (assuming the `main` branch here is the latest), the slowers in `clickbench_1` look like this? > > My script tries to control for this by comparing against `git merge-base` -- FWIW the script I am using is here https://github.com/alamb/datafusion-benchmarking/blob/main/compare_branch.sh > > > * Maybe measurement noise exists? The slower in `tpch_mem_sf1` may be due to this? > > Yes, maybe > > > I am running `clickbench_1` in my local to try reproducing it. > > Than thank you. I also hope to spend some time shortly looking into this (and it looks like you have done some additonal work too) I am still working on finding the reason why q22 (as mentions about the strange result above, after pulling and rebasing to the latest main, this branch took 9200ms, and main 8800ms now... I used perf to collects some cpu metrics, seems almost same ... - latest main ``` 83,416,745,608 dTLB-loads 22,018,889 dTLB-load-misses # 0.03% of all dTLB cache hits <not supported> dTLB-prefetch-misses 83,416,745,608 L1-dcache-loads 6,315,625,925 L1-dcache-load-misses # 7.57% of all L1-dcache hits 29,755,469,086 L1-dcache-stores 17.828162002 seconds time elapsed 91.404349000 seconds user 5.162641000 seconds sys ``` - this branch ``` 83,396,296,754 dTLB-loads 21,873,549 dTLB-load-misses # 0.03% of all dTLB cache hits <not supported> dTLB-prefetch-misses 83,396,296,754 L1-dcache-loads 6,312,881,833 L1-dcache-load-misses # 7.57% of all L1-dcache hits 29,739,091,743 L1-dcache-stores 18.637998137 seconds time elapsed 94.941212000 seconds user 5.033127000 seconds sys ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
