Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/19327#discussion_r140612077
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/SymmetricHashJoinStateManager.scala
---
@@ -329,6 +392,27 @@ class SymmetricHashJoinStateManager(
}
}
+ /** Get all the values for key and all indices, in a (value, index)
tuple. */
+ def getAllWithIndex(key: UnsafeRow, numValues: Long):
Iterator[(UnsafeRow, Long)] = {
--- End diff --
We can probably convert getAll to this. Does not make sense to have both,
especially both are equally efficient if you return
Iterator[KeyWithIndexAndValue].
Also, I think the iterator() method can be removed. its effectively not
being used (used only in StateManager.iterator, which is not being used really).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
