alamb commented on PR #20665: URL: https://github.com/apache/datafusion/pull/20665#issuecomment-4010277050
> `│ QQuery 23 │ 25151.55 ms │ 11723.85 ms │ +2.15x faster │` > > @alamb btw this was the query I was talking about... > > * Is 2-3 times slower 50% of the time in main > * It is _always_ the left/main branch which is slower. > * Sometimes some other queries show similar issues for the main branch (making the overall comparison off) > > When running locally, I don't have the problem... I think because it's the most IO heavy one, it must be something related to IOPs, swapping... @Dandandan I think it is this one: https://github.com/apache/datafusion/blob/b3976d6320a1715b3f7232aaa1f52a8615a28d12/benchmarks/queries/clickbench/queries/q23.sql#L4 Last tiem @adriangb looked into that I think he concluded that a bunch of the variability is related to if the TopK finds the top values quickly enough to skip some of the files entirely but that depends on a race to read the "right" rows. Q23 is also the one that gets stupidly faster (like 20x) when we turn on filter pushdown -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
