Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21467#discussion_r192436670
--- Diff: python/pyspark/sql/tests.py ---
@@ -901,20 +901,43 @@ def __call__(self, x):
self.assertEqual(return_type, f_.returnType)
def test_stopiteration_in_udf(self):
- # test for SPARK-23754
- from pyspark.sql.functions import udf
+ from pyspark.sql.functions import udf, pandas_udf, PandasUDFType
from py4j.protocol import Py4JJavaError
+ def do_test(action, *args, **kwargs):
--- End diff --
Likewise, let's just do
```python
exc_message = "Caught StopIteration thrown from user's code; failing the
task"
with self.assertRaisesRegexp(Py4JJavaError, exc_message):
... pandas_udf ...
```
I think it doesn't particularly reduce the codes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
