Github user e-dorigatti commented on a diff in the pull request:
https://github.com/apache/spark/pull/21467#discussion_r192455582
--- Diff: python/pyspark/worker.py ---
@@ -140,15 +139,20 @@ def read_single_udf(pickleSer, infile, eval_type):
else:
row_func = chain(row_func, f)
+ # make sure StopIteration's raised in the user code are not
+ # ignored, but re-raised as RuntimeError's
+ func = fail_on_stopiteration(row_func)
--- End diff --
I wanted to avoid the overhead of calling get_argspec even when it's not
needed
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
