This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push: new c309cd1 [SPARK-38522][SS] Enrich the method contract of iterator in StateStore to not expect strong consistency on certain condition c309cd1 is described below commit c309cd136a04d831872a83ad425cacc1c557aa60 Author: Jungtaek Lim <kabhwan.opensou...@gmail.com> AuthorDate: Wed Mar 23 10:09:06 2022 +0900 [SPARK-38522][SS] Enrich the method contract of iterator in StateStore to not expect strong consistency on certain condition ### What changes were proposed in this pull request? This PR proposes to enrich the method contract of iterator in StateStore, that the returned iterator is not guaranteed to reflect all updates being performed "after" it has been initialized. ### Why are the changes needed? The lack of information on the method contract could mislead callers to expect some guarantees although not described. It would be nice to add to the contract to make clear. It could also help to implementations that they are not required to guarantee such thing. ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? N/A Closes #35816 from HeartSaVioR/SPARK-38522. Authored-by: Jungtaek Lim <kabhwan.opensou...@gmail.com> Signed-off-by: Jungtaek Lim <kabhwan.opensou...@gmail.com> --- .../org/apache/spark/sql/execution/streaming/state/StateStore.scala | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/StateStore.scala b/sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/StateStore.scala index 5020638..64c9cd1 100644 --- a/sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/StateStore.scala +++ b/sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/StateStore.scala @@ -127,6 +127,10 @@ trait StateStore extends ReadStateStore { /** * Return an iterator containing all the key-value pairs in the StateStore. Implementations must * ensure that updates (puts, removes) can be made while iterating over this iterator. + * + * It is not required for implementations to ensure the iterator reflects all updates being + * performed after initialization of the iterator. Callers should perform all updates before + * calling this method if all updates should be visible in the returned iterator. */ override def iterator(): Iterator[UnsafeRowPair] --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org