gabotechs commented on PR #23975: URL: https://github.com/apache/datafusion/pull/23975#issuecomment-5180843222
@alamb I imagine you are referring to https://github.com/apache/datafusion/pull/23772. Unless I misinterpreted the intentions there, I see that's focus on performance benchmarks, while this PR focuses on planning-time statistics vs execution-time metrics statistical divergence. I've tried several things already for shipping a tool that can qualify improvements to DataFusion stats system: - Modeling stats estimation as integration tests, attempted in https://github.com/apache/datafusion/pull/20292 without success because of non-determinism. - Enhancing existing benchmarks so that they can additionally output the planning stats VS execution metrics as another output. However, this required shoehorning some logic into the existing benchmarks, and the benefit is not very big, as performance-based benchmarks have a different set of requirements that are not needed for planning stats VS execution metrics benchmarks, like executing the same queries a certain amount of times, measuring timing, etc... As there's really not any precedence about what I'm trying to do here, and it does not quite fit neither in the existing benchmark infrastructure or the current integration tests, I preferred to ship something as isolated as possible, so that: - It does not get in the way of normal benchmarks with additional (potentially unrelated) code - We can easily nuke it if we don't find it useful without touching any other pre-existing files. Also @Omega359, if you have ideas about how to integrate this with your work, they are more than welcome, but my impression is that integrating this with the existing or future (https://github.com/apache/datafusion/pull/23772) is going to require some shoehorning that is likely to get in the way of other efforts rather than being helpful. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
