Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21383#discussion_r190450475
--- Diff: python/pyspark/util.py ---
@@ -89,6 +89,19 @@ def majorMinorVersion(sparkVersion):
" version numbers.")
+def fail_on_StopIteration(f):
+ """ wraps f to make it safe (= does not lead to data loss) to use
inside a for loop
--- End diff --
How about something like `Wraps the input function to fail on StopIteration
by RuntimeError to prevent data loss silently.`?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
