uros-b commented on code in PR #57796:
URL: https://github.com/apache/spark/pull/57796#discussion_r3724662831
##########
core/src/main/scala/org/apache/spark/deploy/master/ApplicationInfo.scala:
##########
@@ -204,4 +205,23 @@ private[spark] class ApplicationInfo(
System.currentTimeMillis() - startTime
}
}
+
+ private[deploy] def redactedCopy(conf: SparkConf): ApplicationInfo = {
+ val redactedCommand = desc.command.copy(
+ environment = Utils.redact(conf, desc.command.environment.toSeq).toMap,
+ javaOpts = Utils.redactCommandLineArgs(conf, desc.command.javaOpts))
+ val redactedDesc = desc.copy(command = redactedCommand)
+ new ApplicationInfo(startTime, id, redactedDesc, submitDate, driver,
defaultCores)
+ }
+
+ @transient private var _conf: SparkConf = _
+
+ private[deploy] def withConf(conf: SparkConf): this.type = {
+ _conf = conf
+ this
+ }
+
+ private def writeReplace(): AnyRef = {
+ if (_conf == null) this else redactedCopy(_conf)
+ }
}
Review Comment:
Functional regression in HA recovery for supervised drivers and executors.
After a master crash and recovery, the objects deserialized from disk are
already-redacted copies: desc.command.environment and desc.command.javaOpts
contain Utils.REDACTION_REPLACEMENT_TEXT in place of real secrets.
The recovery path in Master.completeRecovery() calls relaunchDriver(d) for
every supervised driver whose worker is gone, and relaunchDriver calls
createDriver(driver.desc); where driver.desc is the recovered,
permanently-redacted DriverDescription. The resulting DriverInfo (even with
withConf(conf) attached) has a redacted command, so the worker receives
LaunchDriver with REDACTION_REPLACEMENT_TEXT in env vars. Any driver that
requires env-var secrets (e.g. HADOOP_CREDSTORE_PASSWORD, AWS_ACCESS_KEY) will
fail to authenticate after master-crash recovery. The same applies to executor
re-launches: when a recovered application needs new executors, launchExecutor
sends exec.application.desc (the redacted ApplicationDescription) to the
worker, which builds the executor process with the redacted environment. The
PersistenceEngineSuite test explicitly asserts
recoveredApp.desc.command.environment("PASSWORD") ==
Utils.REDACTION_REPLACEMENT_TEXT, confirming the regression.
The fix should store an out-of-band, separate redacted copy for persistence
(e.g., serialize a lightweight ApplicationDescription/DriverDescription
snapshot with redacted fields) rather than having the live deserialized object
carry permanently-redacted state.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
