zhengruifeng opened a new pull request, #58385: URL: https://github.com/apache/spark/pull/58385
### What changes were proposed in this pull request? Document the public SQL configurations that can affect the behavior of functions exposed by `org.apache.spark.sql.functions` and `pyspark.sql.functions`. The updated Scala and Python API documentation covers these configurations: - `spark.sql.ansi.enabled` - `spark.sql.chunkBase64String.enabled` - `spark.sql.columnNameOfCorruptRecord` - `spark.sql.icu.caseMappings.enabled` - `spark.sql.jsonGenerator.ignoreNullFields` - `spark.sql.mapKeyDedupPolicy` - `spark.sql.reflect.allowList` - `spark.sql.session.timeZone` - `spark.sql.timestampType` - `spark.sql.xml.variant.respectInferSchema` ### Why are the changes needed? Some SQL functions change their behavior based on session configuration, but their API documentation did not consistently identify those dependencies. Listing the relevant public configuration keys makes those behavior differences discoverable from both the Scala and Python function documentation. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? No tests were added because this is a documentation-only change. The patch was validated with: - `git diff --check` - Python AST parsing of `python/pyspark/sql/functions/builtin.py` - changed-line checks for the 100-character limit and non-ASCII characters - a coverage check that every identified config-sensitive function is documented in both APIs ### Was this patch authored or co-authored using generative AI tooling? Generated-by: OpenAI Codex (GPT-5) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
