[
https://issues.apache.org/jira/browse/SPARK-23121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16332249#comment-16332249
]
Sandor Murakozi commented on SPARK-23121:
-----------------------------------------
[~guoxiaolongzte] found two separate problems, bot triggered by having a high
number of jobs/stages. In such a situation the store of the history server
drops various objects to save memory. It may happen that the job itself is in
the store, but its stages or the RDDOperationGraph are not. In such cases
rendering of the all jobs and the job pages fails.
As a consequence, the jobs page may become inaccessible if the cluster
processes many jobs, so I think the priority of this issue should be increased.
What do you think [~srowen] ?
> When the Spark Streaming app is running for a period of time, the page is
> incorrectly reported when accessing '/ jobs /' or '/ jobs / job /? Id = 13'
> and ui can not be accessed.
> ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-23121
> URL: https://issues.apache.org/jira/browse/SPARK-23121
> Project: Spark
> Issue Type: Bug
> Components: Web UI
> Affects Versions: 2.4.0
> Reporter: guoxiaolongzte
> Priority: Major
> Attachments: 1.png, 2.png
>
>
> When the Spark Streaming app is running for a period of time, the page is
> incorrectly reported when accessing '/ jobs /' or '/ jobs / job /? Id = 13'
> and ui can not be accessed.
>
> Test command:
> ./bin/spark-submit --class org.apache.spark.examples.streaming.HdfsWordCount
> ./examples/jars/spark-examples_2.11-2.4.0-SNAPSHOT.jar /spark
>
> The app is running for a period of time, ui can not be accessed, please see
> attachment.
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
