Github user HeartSaVioR commented on a diff in the pull request:
https://github.com/apache/spark/pull/21700#discussion_r201562690
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/HDFSBackedStateStoreProvider.scala
---
@@ -239,8 +241,9 @@ private[state] class HDFSBackedStateStoreProvider
extends StateStoreProvider wit
@volatile private var valueSchema: StructType = _
@volatile private var storeConf: StateStoreConf = _
@volatile private var hadoopConf: Configuration = _
+ @volatile private var numberOfVersionsToRetainInMemory: Int = _
- private lazy val loadedMaps = new mutable.HashMap[Long, MapType]
+ private lazy val loadedMaps = new util.TreeMap[Long,
MapType](Ordering[Long].reverse)
--- End diff --
Just FYI: Referring java.util.TreeMap is unavoidable cause Scala doesn't
support mutable SortedMap until Scala 2.12, hence needs additional changes for
interop.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
