gaborgsomogyi commented on a change in pull request #26935:
URL: https://github.com/apache/spark/pull/26935#discussion_r511974992
##########
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/StateStore.scala
##########
@@ -89,16 +116,16 @@ trait StateStore {
def commit(): Long
/**
- * Abort all the updates that have been made to the store. Implementations
should ensure that
- * no more updates (puts, removes) can be after an abort in order to avoid
incorrect usage.
+ * Return an iterator containing all the key-value pairs in the StateStore.
Implementations must
+ * ensure that updates (puts, removes) can be made while iterating over this
iterator.
*/
- def abort(): Unit
+ override def iterator(): Iterator[UnsafeRowPair]
/**
- * Return an iterator containing all the key-value pairs in the StateStore.
Implementations must
- * ensure that updates (puts, removes) can be made while iterating over this
iterator.
+ * Abort all the updates that have been made to the store. Implementations
should ensure that
+ * no more updates (puts, removes) can be after an abort in order to avoid
incorrect usage.
*/
- def iterator(): Iterator[UnsafeRowPair]
+ override def abort(): Unit
Review comment:
Having an API which may or may not need to be called looks super odd.
Not yet sure so asking doesn't this break the `Liskow` substitution law? All
baseclass instances must be replaceable w/ the subclass instance. If I
understand correctly `ReadStateStore.abort` always called to free up some
resources. What happens when `StateStore.abort` called? According to the doc it
may or may not be called (which is good) but does subclass handles a call
correctly? If a sublass fails or makes the behavior different only by calling
`abort` then something is not 100%.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
