Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/16252#discussion_r92477075
--- Diff:
core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala ---
@@ -694,7 +694,7 @@ private[storage] class PartiallyUnrolledIterator[T](
}
override def next(): T = {
- if (unrolled == null) {
+ if (unrolled == null || !unrolled.hasNext) {
--- End diff --
+1. I don't like to expose undefined behavior to the user, either. It's
better to throw an exception instead. E.g., Scala's `ConcatIterator` is also
implemented in this way:
https://github.com/scala/scala/blob/v2.12.1/src/library/scala/collection/Iterator.scala#L216
IMO, we should not assume an Iterator follows Java Iterator's contract, but
if we are implementing, it's better to follow it to avoiding spending a lot of
time on debugging misusing Iterator in future.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
