srielau opened a new pull request, #59467: URL: https://github.com/apache/spark/pull/59467
### What changes were proposed in this pull request? [SPARK-59016](https://issues.apache.org/jira/browse/SPARK-59016) residual after the nested-shape inventory landed: - Broaden the FunctionRegistry CHAR/VARCHAR leak inventory beyond a `Try(...).toOption` probe on a single CHAR(5) column. Cover typecheck-gated signatures (`translate`, `overlay`, `format_string`/`printf`, `split_part`, `substring_index`, HOF lambdas, window `OVER`), VARCHAR inputs, and non-UTF8_BINARY collations. Required probes must typecheck rather than being skipped. - Stop `PushDownUtils.toOutputAttrs` from re-stamping `__CHAR_VARCHAR_TYPE_STRING` after `ApplyCharTypePadding`. Column prune re-derives output from `scan.readSchema()` in the optimizer with no second padding pass. File sources usually already substituted STRING (latent); a V2 scan whose `readSchema()` still surfaces CHAR/VARCHAR would look unpadded. No transforming-function leak was found; the inventory change is an assurance gap. The DSv2 change is a real prune-path hazard for V2 sources that expose CHAR/VARCHAR in `readSchema()`. ### Why are the changes needed? The merged inventory could not fail CI on a leak in a signature that does not typecheck with the original templates. DSv2 prune could restore the "not yet padded" CHAR breadcrumb after the analyzer had already padded. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? - `build/sbt "sql/testOnly org.apache.spark.sql.BasicCharVarcharTestSuite -- -z SPARK-59016"` - `build/sbt "sql/testOnly org.apache.spark.sql.DSV2CharVarcharTestSuite -- -z SPARK-59016"` ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Cursor Grok 4.6 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
