andygrove opened a new pull request, #4244: URL: https://github.com/apache/datafusion-comet/pull/4244
## Which issue does this PR close? Closes #. ## Rationale for this change Several user-facing docs are out of date ahead of the 0.16 release: - The Spark version compatibility, installation, and Gluten comparison pages still describe Spark 4.1 as experimental even though it now runs in full CI alongside Spark SQL tests. - The installation page lists Apple macOS (Intel and Apple Silicon) under supported operating systems without noting that the published Maven jars only ship Linux native binaries; Intel macOS is also not exercised by CI any more. - Several recently-added math expressions are still marked `[ ]` in the contributor-guide tracking page. - `spark.comet.scan.enabled` is documented as a user-facing scan toggle, but in practice it exists to let Comet's own tests selectively disable native scans. - The Iceberg and Kubernetes guides hard-code older Comet versions (`0.14.0` and `0.7.0`) in the example commands and YAML, and the Iceberg example also redundantly sets `spark.sql.extensions=org.apache.comet.CometSparkSessionExtensions`, which `CometPlugin` already registers. ## What changes are included in this PR? - Promote Spark 4.1 to fully supported in `compatibility/spark-versions.md`, `installation.md`, and `about/gluten_comparison.md`. The experimental table now lists only Spark 4.2.0-preview4. Spark 4.1.1 is shown as running on JDK 17/21. - Restructure the Supported Operating Systems section in `installation.md` into a table that distinguishes published Maven jars (Linux amd64/arm64) from build-from-source coverage. Drop the Intel macOS row. - Flip seven `math_funcs` entries (`acosh`, `asinh`, `atanh`, `cbrt`, `degrees`, `pi`, `radians`) from `[ ]` to `[x]` in `docs/source/contributor-guide/spark_expressions_support.md` to reflect commits a4f022939, 1b4b26f31, e5351f4bc, and 356dd944a. - Move `spark.comet.scan.enabled` from `CATEGORY_SCAN` to `CATEGORY_TESTING` in `CometConf.scala` and rewrite the doc string to make clear it is intended for Comet's own test suites. Remove the corresponding paragraph from `datasources.md`. - Replace the hard-coded `0.14.0` references in `iceberg.md` and `0.7.0` references in `kubernetes.md` with the `$COMET_VERSION` placeholder used elsewhere in the docs. Drop the redundant `spark.sql.extensions=org.apache.comet.CometSparkSessionExtensions` conf from both Iceberg spark-shell examples. ## How are these changes tested? Documentation-only changes apart from the `CometConf` doc string and category move; the `CONFIG_TABLE[testing]` section in `configs.md` is regenerated from `CometConf` by `GenerateDocs.scala` on merge. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
