andygrove opened a new pull request, #5197: URL: https://github.com/apache/datafusion-comet/pull/5197
## Which issue does this PR close? Part of #4978. ## Rationale for this change `spark.comet.exec.pyarrowUdf.enabled` and `spark.comet.exec.scalaUDF.codegen.enabled` spell the same acronym two different ways. The [configuration conventions](https://datafusion.apache.org/comet/contributor-guide/config_conventions.html) guide calls this out by name as a red flag, so the guide was citing a live inconsistency in our own config surface. The key was added in #4234 and has not shipped in a release (it is not in 0.17.1), so it can be renamed outright with **no deprecated alias** — which is only true until 1.0 goes out. Doing it now costs nothing; doing it later means carrying a `withAlternative` through all of 1.x. While in the conventions doc I swept it for other references that no longer match the code. ## What changes are included in this PR? Rename (no `withAlternative`, key is unreleased): - `spark.comet.exec.pyarrowUdf.enabled` → `spark.comet.exec.pyarrowUDF.enabled` The Scala symbol `COMET_PYARROW_UDF_ENABLED` was already correct, so all Scala callsites and `CometMapInBatchSuite` are untouched. Only the key string changes, in `CometConf.scala`, one doc comment, the user guide, and the two pyspark scripts that set the key by name. Stale references fixed in `config_conventions.md`: | Claim in the doc | Reality | | --- | --- | | `columnar` is a category segment | No `spark.comet.columnar.*` key remains — unified under `spark.comet.shuffle.*` in #4986 | | (`debug` absent from the category list) | `spark.comet.debug.enabled` exists | | `spark.comet.parquet.rowFilterPushdown.enabled` → `COMET_ROW_FILTER_PUSHDOWN` | Actual symbol is `COMET_PARQUET_ROW_FILTER_PUSHDOWN_ENABLED` | | Acronym rule cites `pyarrowUdf` as mixed-casing example | Fixed by this PR | | Behavior-change section says to use `.category(CATEGORY_LEGACY)` | No legacy config and no `CATEGORY_LEGACY` constant exists yet; now notes the first one to land must add it | ## How are these changes tested? `mvnw compile -Pspark-4.0` (the profile that owns the PyArrow UDF path) is clean, as is `spotless:check`. The rename is a pure key-string change with no behavior change; existing `CometMapInBatchSuite` coverage goes through the Scala symbol and is unaffected. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
