LuciferYang opened a new pull request, #60025: URL: https://github.com/apache/spark/pull/60025
### What changes were proposed in this pull request? `EstimationUtils.getAliasStats` now passes a column's statistics through the read-side CHAR padding alias, `Alias(StaticInvoke(CharVarcharCodegenUtils, "readSidePadding", c, N))`, the same way it already does for a plain `Alias(c)`. ### Why are the changes needed? With `spark.sql.readSideCharPadding` at its default of `true`, `ApplyCharTypePadding` wraps every CHAR column of a scan in that alias. `getAliasStats` only recognized `Alias(Attribute)` and foldable aliases, so a CHAR column lost its statistics at the first `Project` and the estimators that need them, such as `AggregateEstimation`, fell back to size-only estimation. For example, with CBO on, after `ANALYZE TABLE ... COMPUTE STATISTICS FOR COLUMNS` on a `CHAR(5)` column holding 10 distinct values in 1000 rows, the `Aggregate` of `SELECT c, count(*) FROM t GROUP BY c` has no row count estimate. The same query on a `STRING` column estimates 10. Passing the statistics through unchanged is exact, because `ANALYZE TABLE` reads the table through the same padding and the stored statistics already describe padded values: an external `CHAR(3)` table holding the raw values `"a"`, `"a "` and `"b"` is analyzed as 2 distinct values with a max length of 3. For string types `ANALYZE` stores no min, max or histogram, so nothing else in the `ColumnStat` depends on the padding. If a table was analyzed with read-side padding off and is read with it on, the distinct count can be too high, which only affects the estimate. The TPC-DS plan stability suites did not catch this because they turn read-side padding off. ### Does this PR introduce _any_ user-facing change? No. Query results are unchanged. With CBO enabled, plans over analyzed CHAR columns can change because the estimators now see the columns' statistics. ### How was this patch tested? New tests, both of which fail without the change: - `ProjectEstimationSuite`: a `Project` over the read-side padding alias of a `CHAR(5)` attribute keeps the attribute's `ColumnStat`. - `StatisticsCollectionSuite`: after `ANALYZE` on a `CHAR(5)` column, `GROUP BY` on it is estimated at 10 rows under CBO. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Opus 5.5 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
