Yicong-Huang opened a new pull request, #59774: URL: https://github.com/apache/spark/pull/59774
### What changes were proposed in this pull request? Enable the default `fail_on_stopiteration` wrap for `SQL_SCALAR_PANDAS_ITER_UDF` (`PandasScalarIterUDFHandler`) and `SQL_SCALAR_PANDAS_ITER_ELEMENTWISE_UDF` (`PandasScalarIterElementwiseUDFHandler`), matching their Arrow counterparts, and correct the `_fail_on_stopiteration` comment: since `run` is a generator, a leaked StopIteration already fails the task via PEP 479; the wrap only turns it into a clear `STOP_ITERATION_OCCURRED`. ### Why are the changes needed? Consistent error across the scalar iterator handlers; the old comment overstated the wrap's effect. ### Does this PR introduce _any_ user-facing change? Yes, small. A StopIteration raised by the UDF call itself (before it returns an iterator) now fails with `PySparkRuntimeError [STOP_ITERATION_OCCURRED]` instead of `RuntimeError: generator raised StopIteration`. StopIteration raised while iterating the returned iterator is unchanged. ### How was this patch tested? New unit tests in `test_pandas_eval_type_handlers.py` / `test_elementwise_eval_type_handlers.py`; updated `test_fail_on_stopiteration_per_handler`. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Opus 5.5 This pull request and its description were written by Isaac. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
