[
https://issues.apache.org/jira/browse/SPARK-58600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yicong Huang resolved SPARK-58600.
----------------------------------
Resolution: Won't Do
Closing as Won't Do. Investigation showed the premise does not hold:
python/docs/source/development/errors.rst is not a tracked file. It has been
listed in .gitignore since 2023 (commit 2b731565941, right after SPARK-44945
automated the doc) and is generated at documentation build time by
python/docs/source/conf.py calling pyspark.errors_doc_gen.generate_errors_doc.
Re-running the generator on current master leaves the working tree clean (the
output is gitignored), and the freshly generated content already reflects
error-conditions.json exactly (the 7 stale classes are absent and the 5 missing
classes are present). The apparent drift only existed in a stale local copy of
errors.rst from a previous build, not in anything published or
version-controlled. No fix or PR is warranted.
> Regenerate PySpark error-class documentation to fix drift from
> error-conditions.json
> ------------------------------------------------------------------------------------
>
> Key: SPARK-58600
> URL: https://issues.apache.org/jira/browse/SPARK-58600
> Project: Spark
> Issue Type: Documentation
> Components: PySpark
> Affects Versions: 4.4.0
> Reporter: Yicong Huang
> Priority: Major
>
> The auto-generated PySpark error-class doc
> (python/docs/source/development/errors.rst) has drifted from
> python/pyspark/errors/error-conditions.json. Several PRs added or removed
> error classes by editing error-conditions.json without re-running the
> generator (python/pyspark/errors_doc_gen.py, introduced in SPARK-44945), so
> the published doc is stale.
> Verified drift on current master:
> Listed in errors.rst but no longer in the JSON (stale, must be removed):
> * SCHEMA_MISMATCH_FOR_PANDAS_UDF
> * RESULT_TYPE_MISMATCH_FOR_ARROW_UDF
> * SCHEMA_MISMATCH_FOR_ARROW_PYTHON_UDF
> * UNKNOWN_EXPLAIN_MODE
> * VALUE_NOT_ANY_OR_ALL
> * VALUE_NOT_PEARSON
> * VALUE_NOT_TRUE
> In the JSON but missing from errors.rst (must be added):
> * LOCAL_CONNECT_RUNTIME_DIR_UNAVAILABLE
> * LOCAL_CONNECT_SERVER_START_FAILED
> * NEAREST_BY_JOIN
> * RESULT_COLUMN_TYPES_MISMATCH
> * ST_INVALID_ENDIANNESS_VALUE
> Fix: re-run the generator and commit the result (the file is auto-generated
> and should not be hand-edited).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
