[
https://issues.apache.org/jira/browse/SPARK-37562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17747676#comment-17747676
]
Holden Karau commented on SPARK-37562:
--------------------------------------
So (in theory) the cluster administrator has some base config, they set it up.
They also configure a history server location. When we run on YARN they can
configure that location the the log URL will be printed with the correct
location (e.g. [historyserver]/[app] ) for someone investigating after the case.
This just proposes to generalize the YARN config.
> Add Spark History Server Links for Kubernetes & other CMs
> ---------------------------------------------------------
>
> Key: SPARK-37562
> URL: https://issues.apache.org/jira/browse/SPARK-37562
> Project: Spark
> Issue Type: Improvement
> Components: Kubernetes
> Affects Versions: 3.2.1, 3.3.0
> Reporter: Holden Karau
> Assignee: Holden Karau
> Priority: Minor
>
> In YARN we have the Spark history server configured with
> `spark.yarn.historyServer.address` which allows us to print out useful links
> on startup for eventual debugging. More than just YARN can have the history
> server. We should either add `spark.kubernetes.historyServer.address` or move
> it to `spark.historyServer.address` w/a fall back to the old YARN specific
> config.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
