dongjoon-hyun commented on code in PR #44173:

@@ -67,6 +67,25 @@ private[master] class FileSystemRecoveryModeFactory(conf: 
SparkConf, serializer:
+ * LeaderAgent in this case is a no-op. Since leader is forever leader as the 
+ * recovery is made by restoring from RocksDB.
+ */
+private[master] class RocksDBRecoveryModeFactory(conf: SparkConf, serializer: 
+  extends StandaloneRecoveryModeFactory(conf, serializer) with Logging {
+  val recoveryDir = conf.get(RECOVERY_DIRECTORY)

Review Comment:
   Thank you for review. @yaooqinn and @LuciferYang 
   I tried according to your advices but it seems to make the configuration 
namespace a little weird because this existing configuration is a general one, 
   If we introduce a new configuration for `RocksDBPersistenceEngine` like the 
following, the users will ask why we cannot use 
`spark.deploy.recoveryDirectory` for `ROCKSDB` mode. Here is the example. The 
AS-IS design aims to allow the users to switch a single configuration 
   If we add a new configuration, it would be the following where 
`spark.deploy.recoveryDirectory` is no-op.
   In addition, we assume that the users clear the location properly when they 
changes the settings like the following. 
   - `spark.deploy.recoverySerializer`
   - `spark.deploy.recoveryCompressionCodec`
   To sum up, if you guys don't mind, I'd like to use the existing general 
configuration name for this backend.

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail:

For queries about this service, please contact Infrastructure at:

To unsubscribe, e-mail:
For additional commands, e-mail:

Reply via email to