Alwaysgaurav1 opened a new pull request, #57204:
URL: https://github.com/apache/spark/pull/57204
### What changes were proposed in this pull request?
This PR resolves a TODO in `python/pyspark/worker.py` to standardize row
count mismatch error reporting by using the unified `RESULT_ROWS_MISMATCH`
error class instead of the legacy `SCHEMA_MISMATCH_FOR_PANDAS_UDF` class.
### Why are the changes needed?
Previously, PySpark raised a custom `SCHEMA_MISMATCH_FOR_PANDAS_UDF` error
class during UDF row count mismatches. Standardizing this error class to
`RESULT_ROWS_MISMATCH` makes error codes and message patterns uniform across
PySpark.
### Does this PR introduce _any_ user-facing change?
Yes, the error class raised on a Pandas/Arrow UDF row count mismatch changes
from `SCHEMA_MISMATCH_FOR_PANDAS_UDF` to `RESULT_ROWS_MISMATCH`.
### How was this patch tested?
1. Ran unit tests targeting Pandas UDF execution:
```bash
python/run-tests.py --python-executables venv/bin/python --testnames
pyspark.sql.tests.pandas.test_pandas_udf
```
**Result**: Passed successfully.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
