Chirag Jain created SPARK-52496:
-----------------------------------
Summary: Redirect do not honor proxyBase in Completed Batches
Table links
Key: SPARK-52496
URL: https://issues.apache.org/jira/browse/SPARK-52496
Project: Spark
Issue Type: Bug
Components: UI, Web UI
Affects Versions: 3.5.3
Reporter: Chirag Jain
We are running a streaming SparkApplication on K8s with Istio
We have the following properties set
```
spark.ui.enabled: 'true'
spark.ui.proxyBase: /spark-ui/spark-history-e2e-test-cd7d74179813
spark.ui.proxyRedirectUri: /
```
The spark web ui loads and works fine at
https://spark.example.com/spark-ui/spark-history-e2e-test-cd7d74179813/jobs
Except for the Completed Batches Table under Streaming - the links in there do
not have a trailing slash
[https://github.com/apache/spark/blob/59e6b5b7d350a1603502bc92e3c117311ab2cbb6/streaming/src/main/scala/org/apache/spark/streaming/ui/AllBatchesTable.scala#L148]
which leads to Jetty redirecting and adding a '/', however we notice the
proxyBase is not being honored in this redirect
{code:java}
$ curl -L -I
https://spark.example.com/spark-ui/spark-history-e2e-test-cd7d74179813/streaming/batch?id=1750103432000
HTTP/2 302
date: Mon, 16 Jun 2025 19:54:22 GMT
location: http://spark.example.com/streaming/batch/?id=1750103432000
content-length: 0
x-envoy-upstream-service-time: 2
server: istio-envoy
HTTP/1.1 301 Moved Permanently
location: https://spark.example.com/streaming/batch/?id=1750103432000
date: Mon, 16 Jun 2025 19:54:22 GMT
server: istio-envoy
transfer-encoding: chunked
HTTP/2 404
date: Mon, 16 Jun 2025 19:54:22 GMT
server: istio-envoy
{code}
As we see `/spark-ui/spark-history-e2e-test-cd7d74179813` is dropped.
---
A short term fix is to make the batch links render with the proxyBase and a
trailing slash
But I am puzzled why ProxyRedirectHandler does not honor proxyBase - the code
seems to be written to handle it
https://github.com/apache/spark/blob/32232e9ed33bb16b93ad58cfde8b82e0f07c0970/core/src/main/scala/org/apache/spark/ui/JettyUtils.scala#L602-L603
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
