Shiv-61 opened a new pull request, #59983: URL: https://github.com/apache/spark/pull/59983
### What changes were proposed in this pull request? This PR addresses https://github.com/apache/spark/issues/59826 by making built-in SQL function documentation generation deterministic: 1. In `PythonSQLUtils.listBuiltinFunctionInfos()`, sort candidates deterministically with `.minBy(...)` on `(className, usage, examples)` for functions present in both `FunctionRegistry` and `TableFunctionRegistry` (e.g., `explode`, `inline`, `posexplode`), and sort the output array by function name. 2. In `sql/gen-sql-functions-docs.py` and `sql/gen-sql-api-docs.py`, use composite sort keys with `usage` and `examples` as stable tie-breakers. 3. Added a regression test in `ExpressionInfoSuite`. ### Why are the changes needed? Fixes #59826. When multiple `ExpressionInfo` entries share the same function name across `FunctionRegistry` and `TableFunctionRegistry`, `PythonSQLUtils.listBuiltinFunctionInfos()` previously used `.groupBy(_.getName).map(v => v._2.head)`. Because `functionSet` is an unordered `Set`, `.head` picked an arbitrary `ExpressionInfo` on fresh JVM runs. This caused generated documentation examples for generator functions (such as `explode`, `inline`, `posexplode`) to differ across clean doc-generation runs. ### Does this PR introduce _any_ user-facing change? No user-facing SQL API change; only fixes non-deterministic SQL documentation generation. ### How was this patch tested? - Added unit test in `ExpressionInfoSuite.scala` verifying deterministic output, sorting, and duplicate resolution for `listBuiltinFunctionInfos()`. - Verified line lengths (<= 100 characters) and confirmed no non-ASCII characters were introduced. Closes #59826. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
