tgravescs commented on code in PR #36200:
URL: https://github.com/apache/spark/pull/36200#discussion_r945874439
##########
common/network-yarn/src/main/java/org/apache/spark/network/yarn/YarnShuffleService.java:
##########
@@ -118,10 +121,10 @@ public class YarnShuffleService extends AuxiliaryService {
private static final String SPARK_AUTHENTICATE_KEY = "spark.authenticate";
private static final boolean DEFAULT_SPARK_AUTHENTICATE = false;
- private static final String RECOVERY_FILE_NAME = "registeredExecutors.ldb";
- private static final String SECRETS_RECOVERY_FILE_NAME =
"sparkShuffleRecovery.ldb";
+ private static final String RECOVERY_FILE_NAME = "registeredExecutors";
Review Comment:
no, sorry, level db should keep .ldb for backwards compatible. If someone
switches to rocksdb, I assume its going to break the upgrade path with support
for rolling restart because the database name would change and all the previous
data stored in leveldb would not be read by rocksdb. personally I think this
is acceptable we just need to document it somewhere. I guess someone could
convert ldb to rocksdb on startup but I think we could wait for someone to
request this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
