shrivasshankar commented on PR #27148: URL: https://github.com/apache/datafusion/pull/27148#issuecomment-6693781289
> @kosiew @xudong963 Are you able to kick off the automated benchmarks? We should run all available benchmarks both with pushdown filtering on and off. If we can think of any other configurations which may be impacted by this as well please run those as well. > > @shrivasshankar > > > Better estimates for range filters on decimal columns with min/max stats. One new public function, check_statistics_support. > > Better estimates is nice! But also, it would be cool to see metrics associated with performance goodness. If you have them please add them to the PR description :) Thanks! With this PR, every TPC-H (SF1, SF10) and TPC-DS (SF1) plan is identical to main, with pushdown_filters both on and off, so I wouldn’t expect runtime changes on those benchmarks. The gain for now is in the estimates themselves. For example, the Q6-style filter goes from 1.2M estimated rows to 111K (actual 114K). I’ll add the benchmark results to the description once they’ve run. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
