gabotechs opened a new issue, #25571: URL: https://github.com/apache/datafusion/issues/25571
### Is your feature request related to a problem or challenge? DataFusion has statistics estimation inside individual operators and separate implementations in the bundled `StatisticsProvider`s. Enabling those providers can replace an operator's estimate with a different calculation for the same operation. This surfaced in #25570: for TPC-H SF1 Q14, the bundled providers produced a join estimate of 14.73 billion rows instead of 73,650, against 75,983 actual rows. Calling the optional set “built-in providers” also makes it hard to tell which implementation is used by default. ### Describe the solution you'd like Could we consolidate built-in statistics estimation so each operator has one authoritative implementation? The registry makes sense as an extension point for custom estimators, but DataFusion's own providers should share the operator's estimation logic rather than maintain a separate algorithm. This would make it clearer where to fix estimation bugs and avoid improvements reaching only one implementation. ### Describe alternatives you've considered Renaming or documenting the optional providers would clarify their behavior, but would leave the duplicated estimation logic in place. ### Additional context #25570 aligns the CLI with the session's configuration. This issue is about consolidating the underlying estimators. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
