gaborgsomogyi commented on a change in pull request #26935:
URL: https://github.com/apache/spark/pull/26935#discussion_r511989317
##########
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/HDFSBackedStateStoreProvider.scala
##########
@@ -197,15 +212,26 @@ private[state] class HDFSBackedStateStoreProvider extends
StateStoreProvider wit
}
/** Get the state store for making updates to create a new `version` of the
store. */
- override def getStore(version: Long): StateStore = synchronized {
+ override def getStore(version: Long): StateStore = {
+ val newMap = getLoadedMapForStore(version)
+ logInfo(s"Retrieved version $version of
${HDFSBackedStateStoreProvider.this} for update")
+ new HDFSBackedStateStore(version, newMap)
+ }
+
+ /** Get the state store for reading to specific `version` of the store. */
+ override def getReadOnlyStore(version: Long): ReadStateStore = {
Review comment:
Just seen that the provider is also `...ReadOnly...`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
