[
https://issues.apache.org/jira/browse/SPARK-23608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16387162#comment-16387162
]
Ye Zhou commented on SPARK-23608:
---------------------------------
I will post a pull request for this minor change.
[~vanzin] [~zsxwing] Any comments? Thanks.
> SHS needs synchronization between attachSparkUI and detachSparkUI functions
> ---------------------------------------------------------------------------
>
> Key: SPARK-23608
> URL: https://issues.apache.org/jira/browse/SPARK-23608
> Project: Spark
> Issue Type: Bug
> Components: Spark Core, Web UI
> Affects Versions: 2.3.0
> Reporter: Ye Zhou
> Priority: Minor
>
> We continuously hit an issue with SHS after it runs for a while and have some
> REST API calls to it. SHS suddenly shows an empty home page with 0
> application. It is caused by the unexpected JSON data returned from rest call
> "api/v1/applications?limit=8000". This REST call returns the home page html
> codes instead of list of application summary. Some other REST call which asks
> for application detailed information also returns home page html codes. But
> there are still some working REST calls. We have to restart SHS to solve the
> issue.
> We attached remote debugger to the problematic process and checked the
> attached jetty handlers tree in the web server. We found that the jetty
> handler added by "attachHandler(ApiRootResource.getServletHandler(this))" is
> not in the tree as well as some other handlers. Without the root resource
> servlet handler, SHS will not work correctly serving both UI and REST calls.
> SHS will directly return the HistoryServerPage html to user as it cannot find
> handlers to handle the request.
> Spark History Server has to attachSparkUI in order to serve user requests.
> The application SparkUI getting attached when the application details data
> gets loaded into Guava Cache. While attaching SparkUI, SHS will add attach
> all jetty handlers into the current web service. But while the data gets
> cleared out from Guava Cache, SHS will detach all the application's SparkUI
> jetty handlers. Due to the asynchronous feature in Guava Cache, the clear out
> from cache is not synchronized with loading into cache. The actual clear out
> in Guava Cache which triggers detachSparkUI might be detaching the handlers
> while the attachSparkUI is attaching jetty handlers.
> After adding synchronization between attachSparkUI and detachSparkUI in
> history server, this issue never happens again.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
