nchammas opened a new pull request, #44300: URL: https://github.com/apache/spark/pull/44300
### What changes were proposed in this pull request? Assign the various SQL configs to groups that can then be referenced in our documentation. This will replace the large number of manually maintained HTML tables with references to generated tables. For example, the SQL performance tuning page has a section on data caching that references two configs. They are in an HTML table that has to be [manually maintained][1]. [1]: https://github.com/apache/spark/blob/7db85642600b1e3b39ca11e41d4e3e0bf1c8962b/docs/sql-performance-tuning.md#L37-L56 With this new system, we can maintain the same documentation output, but instead reference a generated HTML table as follows: ```md {% include_relative generated-sql-config-table-caching-data.html %} ``` And in `SQLConf.scala`, we can assign the configs we want to show up in this table to the `caching-data` group: ```scala val COMPRESS_CACHED = buildConf("spark.sql.inMemoryColumnarStorage.compressed") .doc("When set to true Spark SQL will automatically select a compression codec for each " + "column based on statistics of the data.") .version("1.0.1") .withTag("caching-data") ``` This PR also adds anchors to each config in the generated HTML tables, so that people can link directly to specific configurations in the documentation. <img width="500" alt="Screenshot 2023-12-11 at 4 13 24 PM" src="https://github.com/apache/spark/assets/1039369/5be2743d-c579-4f23-8b84-0cddcb2b0c91";> This PR builds on the work done in #27459, and is related to the work done in #28274. ### Why are the changes needed? To eliminate the need to manually maintain HTML tables for SQL configurations. ### Does this PR introduce _any_ user-facing change? Yes, it alters some of the user-facing documentation. ### How was this patch tested? Manually built the documentation and viewed it in my browser. ### Was this patch authored or co-authored using generative AI tooling? No. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
