Github user e-dorigatti commented on the issue:
https://github.com/apache/spark/pull/21467
@viirya for RDDs, we cannot check for stopiterations in the executor
because this bug is introduced in the driver by rewriting the methods. consider
rdd.map:
```
def map(self, f, preservesPartitioning=False):
def func(_, iterator):
- return map(f, iterator)
+ return map(fail_on_stopiteration(f), iterator)
return self.mapPartitionsWithIndex(func, preservesPartitioning)
```
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
