anishshri-db commented on code in PR #47730:
URL: https://github.com/apache/spark/pull/47730#discussion_r1718827858
##########
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/RocksDB.scala:
##########
@@ -482,12 +491,20 @@ class RocksDB(
}
/**
+ * @param columnFamilyMapping A map of column family names to their
corresponding VCF IDs.
+ * @param maxColumnFamilyId The maximum VCF ID used for column families.
+ * @param colFamilyIdsChanged A flag indicating whether column family IDs
have changed.
+ * If they have, we need to create a snapshot
regardless of whether
+ * changelog checkpointing is enabled or not
* Commit all the updates made as a version to DFS. The steps it needs to do
to commits are:
* - Flush all changes to disk
* - Create a RocksDB checkpoint in a new local dir
* - Sync the checkpoint dir files to DFS
*/
- def commit(): Long = {
+ def commit(
+ columnFamilyMapping: Map[String, Short] = Map.empty,
+ maxColumnFamilyId: Short = 0,
+ colFamilyIdsChanged: Boolean = false): Long = {
Review Comment:
lets rename as `forceSnapshot` ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
