EnricoMi commented on code in PR #45728:
URL: https://github.com/apache/spark/pull/45728#discussion_r1758468322
##########
core/src/main/scala/org/apache/spark/internal/config/UI.scala:
##########
@@ -220,8 +220,20 @@ private[spark] object UI {
.stringConf
.createOptional
+ val CUSTOM_DRIVER_LOG_URL = ConfigBuilder("spark.ui.custom.driver.log.url")
+ .doc("Specifies custom Spark driver log url for supporting external log
service instead of " +
+ "using cluster managers' application log urls in the Spark UI. Spark
will support " +
+ "some path variables via patterns which can vary on cluster manager.
Please check the " +
+ "documentation for your cluster manager to see which patterns are
supported, if any. " +
+ "This configuration replaces original log urls in event log, which will
be also effective " +
+ "when accessing the application on history server. The new log urls must
be permanent, " +
+ "otherwise you might have dead link for executor log urls.")
+ .version("4.0.0")
+ .stringConf
+ .createOptional
+
val CUSTOM_EXECUTOR_LOG_URL =
ConfigBuilder("spark.ui.custom.executor.log.url")
- .doc("Specifies custom spark executor log url for supporting external log
service instead of " +
+ .doc("Specifies custom Spark executor log url for supporting external log
service instead of " +
Review Comment:
done
##########
docs/configuration.md:
##########
@@ -1654,11 +1654,25 @@ Apart from these, the following properties are also
available, and may be useful
</td>
<td>2.1.0</td>
</tr>
+<tr>
+ <td><code>spark.ui.custom.driver.log.url</code></td>
+ <td>(none)</td>
+ <td>
+ Specifies custom Spark driver log URL for supporting external log service
instead of using cluster
+ managers' application log URLs in Spark UI. Spark will support some path
variables via patterns
+ which can vary on cluster manager. Please check the documentation for your
cluster manager to
+ see which patterns are supported, if any. <p/>
+ Please note that this configuration also replaces original log urls in
event log,
+ which will be also effective when accessing the application on history
server. The new log urls must be
+ permanent, otherwise you might have dead link for driver log urls.
+ </td>
+ <td>4.0.0</td>
+</tr>
<tr>
<td><code>spark.ui.custom.executor.log.url</code></td>
<td>(none)</td>
<td>
- Specifies custom spark executor log URL for supporting external log
service instead of using cluster
+ Specifies custom Spark executor log URL for supporting external log
service instead of using cluster
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
