[
https://issues.apache.org/jira/browse/SPARK-41365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mars updated SPARK-41365:
-------------------------
Description:
My environment CDH 5.8 , click to enter the spark UI from the yarn interface
when visit the stage URI, it fails to load, URI is
{code:java}
http://<yarn-url>:8088/proxy/application_1669877165233_0021/stages/stage/?id=0&attempt=0
{code}
!image-2022-12-02-17-53-03-003.png|width=430,height=697!
Server error stack trace:
{code:java}
Caused by: java.lang.NullPointerException
at
org.apache.spark.status.api.v1.StagesResource.$anonfun$doPagination$1(StagesResource.scala:207)
at
org.apache.spark.status.api.v1.BaseAppResource.$anonfun$withUI$1(ApiRootResource.scala:142)
at org.apache.spark.ui.SparkUI.withSparkUI(SparkUI.scala:147)
at
org.apache.spark.status.api.v1.BaseAppResource.withUI(ApiRootResource.scala:137)
at
org.apache.spark.status.api.v1.BaseAppResource.withUI$(ApiRootResource.scala:135)
at
org.apache.spark.status.api.v1.StagesResource.withUI(StagesResource.scala:31)
at
org.apache.spark.status.api.v1.StagesResource.doPagination(StagesResource.scala:206)
at
org.apache.spark.status.api.v1.StagesResource.$anonfun$taskTable$1(StagesResource.scala:161)
at
org.apache.spark.status.api.v1.BaseAppResource.$anonfun$withUI$1(ApiRootResource.scala:142)
at org.apache.spark.ui.SparkUI.withSparkUI(SparkUI.scala:147)
at
org.apache.spark.status.api.v1.BaseAppResource.withUI(ApiRootResource.scala:137)
at
org.apache.spark.status.api.v1.BaseAppResource.withUI$(ApiRootResource.scala:135)
at
org.apache.spark.status.api.v1.StagesResource.withUI(StagesResource.scala:31)
at
org.apache.spark.status.api.v1.StagesResource.taskTable(StagesResource.scala:145)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62){code}
The issue is similar to, the final phenomenon of the issue is the same, because
the parameter encode twice
https://issues.apache.org/jira/browse/SPARK-32467
https://issues.apache.org/jira/browse/SPARK-33611
The two issues solve two scenarios to avoid encode twice:
1. https redirect proxy
2. set reverse proxy enabled (spark.ui.reverseProxy) in Nginx
But if encode twice due to other reasons, such as this issue (yarn proxy), it
will also fail
was:
My environment CDH 5.8 , click to enter the spark UI from the yarn interface
when visit the stage URI, it fails to load, URI is
{code:java}
http://<yarn-url>:8088/proxy/application_1669877165233_0021/stages/stage/?id=0&attempt=0
{code}
!image-2022-12-02-17-53-03-003.png|width=430,height=697!
Server error stack trace:
{code}
Caused by: java.lang.NullPointerException
at
org.apache.spark.status.api.v1.StagesResource.$anonfun$doPagination$1(StagesResource.scala:207)
at
org.apache.spark.status.api.v1.BaseAppResource.$anonfun$withUI$1(ApiRootResource.scala:142)
at org.apache.spark.ui.SparkUI.withSparkUI(SparkUI.scala:147)
at
org.apache.spark.status.api.v1.BaseAppResource.withUI(ApiRootResource.scala:137)
at
org.apache.spark.status.api.v1.BaseAppResource.withUI$(ApiRootResource.scala:135)
at
org.apache.spark.status.api.v1.StagesResource.withUI(StagesResource.scala:31)
at
org.apache.spark.status.api.v1.StagesResource.doPagination(StagesResource.scala:206)
at
org.apache.spark.status.api.v1.StagesResource.$anonfun$taskTable$1(StagesResource.scala:161)
at
org.apache.spark.status.api.v1.BaseAppResource.$anonfun$withUI$1(ApiRootResource.scala:142)
at org.apache.spark.ui.SparkUI.withSparkUI(SparkUI.scala:147)
at
org.apache.spark.status.api.v1.BaseAppResource.withUI(ApiRootResource.scala:137)
at
org.apache.spark.status.api.v1.BaseAppResource.withUI$(ApiRootResource.scala:135)
at
org.apache.spark.status.api.v1.StagesResource.withUI(StagesResource.scala:31)
at
org.apache.spark.status.api.v1.StagesResource.taskTable(StagesResource.scala:145)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62){code}
> Stages UI page fails to load for proxy in some yarn versions
> -------------------------------------------------------------
>
> Key: SPARK-41365
> URL: https://issues.apache.org/jira/browse/SPARK-41365
> Project: Spark
> Issue Type: Bug
> Components: Web UI
> Affects Versions: 3.3.1
> Environment: as above
> Reporter: Mars
> Priority: Major
> Attachments: image-2022-12-02-17-53-03-003.png
>
>
> My environment CDH 5.8 , click to enter the spark UI from the yarn interface
> when visit the stage URI, it fails to load, URI is
> {code:java}
> http://<yarn-url>:8088/proxy/application_1669877165233_0021/stages/stage/?id=0&attempt=0
> {code}
> !image-2022-12-02-17-53-03-003.png|width=430,height=697!
> Server error stack trace:
> {code:java}
> Caused by: java.lang.NullPointerException
> at
> org.apache.spark.status.api.v1.StagesResource.$anonfun$doPagination$1(StagesResource.scala:207)
> at
> org.apache.spark.status.api.v1.BaseAppResource.$anonfun$withUI$1(ApiRootResource.scala:142)
> at org.apache.spark.ui.SparkUI.withSparkUI(SparkUI.scala:147)
> at
> org.apache.spark.status.api.v1.BaseAppResource.withUI(ApiRootResource.scala:137)
> at
> org.apache.spark.status.api.v1.BaseAppResource.withUI$(ApiRootResource.scala:135)
> at
> org.apache.spark.status.api.v1.StagesResource.withUI(StagesResource.scala:31)
> at
> org.apache.spark.status.api.v1.StagesResource.doPagination(StagesResource.scala:206)
> at
> org.apache.spark.status.api.v1.StagesResource.$anonfun$taskTable$1(StagesResource.scala:161)
> at
> org.apache.spark.status.api.v1.BaseAppResource.$anonfun$withUI$1(ApiRootResource.scala:142)
> at org.apache.spark.ui.SparkUI.withSparkUI(SparkUI.scala:147)
> at
> org.apache.spark.status.api.v1.BaseAppResource.withUI(ApiRootResource.scala:137)
> at
> org.apache.spark.status.api.v1.BaseAppResource.withUI$(ApiRootResource.scala:135)
> at
> org.apache.spark.status.api.v1.StagesResource.withUI(StagesResource.scala:31)
> at
> org.apache.spark.status.api.v1.StagesResource.taskTable(StagesResource.scala:145)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62){code}
>
> The issue is similar to, the final phenomenon of the issue is the same,
> because the parameter encode twice
> https://issues.apache.org/jira/browse/SPARK-32467
> https://issues.apache.org/jira/browse/SPARK-33611
> The two issues solve two scenarios to avoid encode twice:
> 1. https redirect proxy
> 2. set reverse proxy enabled (spark.ui.reverseProxy) in Nginx
> But if encode twice due to other reasons, such as this issue (yarn proxy), it
> will also fail
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
