squito commented on a change in pull request #23260: [SPARK-26311][CORE] New
feature: apply custom log URL pattern for executor log URLs in SHS
URL: https://github.com/apache/spark/pull/23260#discussion_r249827598
##########
File path: core/src/main/scala/org/apache/spark/internal/config/History.scala
##########
@@ -125,4 +125,18 @@ private[spark] object History {
val KERBEROS_KEYTAB = ConfigBuilder("spark.history.kerberos.keytab")
.stringConf
.createOptional
+
+ val CUSTOM_EXECUTOR_LOG_URL =
ConfigBuilder("spark.history.custom.executor.log.url")
+ .doc("Specifies custom spark executor log url for supporting external log
service instead of " +
+ "using resource managers' app log urls. Spark will support some path
variables via " +
+ "patterns which can be vary on resource managers. Please check the
document for " +
+ "resource manager to refer which patterns are supported.")
+ .stringConf
+ .createOptional
+
+ val APPLY_CUSTOM_EXECUTOR_LOG_URL_TO_INCOMPLETE_APP =
+
ConfigBuilder("spark.history.custom.executor.log.url.applyIncompleteApplication")
+ .doc("Whether apply custom executor log url to incomplete application as
well")
Review comment:
Whether apply custom executor log url, as specified by
"spark.history.custom.executor.log.url", to incomplete application as well.
Even if this is true, this still only affects the behavior of the history
server, not running spark applications.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
