squito commented on a change in pull request #23260: [SPARK-26311][CORE] New
feature: apply custom log URL pattern for executor log URLs in SHS
URL: https://github.com/apache/spark/pull/23260#discussion_r249954407
##########
File path: core/src/main/scala/org/apache/spark/internal/config/History.scala
##########
@@ -125,4 +125,22 @@ private[spark] object History {
val KERBEROS_KEYTAB = ConfigBuilder("spark.history.kerberos.keytab")
.stringConf
.createOptional
+
+ val CUSTOM_EXECUTOR_LOG_URL =
ConfigBuilder("spark.history.custom.executor.log.url")
+ .doc("Specifies custom spark executor log url for supporting external log
service instead of " +
+ "using cluster managers' application log urls in the history server.
Spark will support " +
+ "some path variables via patterns which can vary on cluster manager.
Please check the " +
+ "documentation for your each cluster manager to see which patterns are
supported, if any. " +
+ "This configuration has no effect on a live application, it only affects
the history server.")
+ .stringConf
+ .createOptional
+
+ val APPLY_CUSTOM_EXECUTOR_LOG_URL_TO_INCOMPLETE_APP =
+
ConfigBuilder("spark.history.custom.executor.log.url.applyIncompleteApplication")
+ .doc("Whether apply custom executor log url, as specified by " +
Review comment:
and one more little nit: "Whether to apply"
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
