[
https://issues.apache.org/jira/browse/SPARK-17672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Or resolved SPARK-17672.
-------------------------------
Resolution: Fixed
Fix Version/s: 2.0.1
Target Version/s: 2.0.1
> Spark 2.0 history server web Ui takes too long for a single application
> -----------------------------------------------------------------------
>
> Key: SPARK-17672
> URL: https://issues.apache.org/jira/browse/SPARK-17672
> Project: Spark
> Issue Type: Bug
> Components: Web UI
> Affects Versions: 2.0.0
> Reporter: Gang Wu
> Fix For: 2.0.1
>
>
> When there are 10K application history in the history server back end, it can
> take a very long time to even get a single application history page. After
> some investigation, I found the root cause was the following piece of code:
> {code:title=OneApplicationResource.scala|borderStyle=solid}
> @Produces(Array(MediaType.APPLICATION_JSON))
> private[v1] class OneApplicationResource(uiRoot: UIRoot) {
> @GET
> def getApp(@PathParam("appId") appId: String): ApplicationInfo = {
> val apps = uiRoot.getApplicationInfoList.find { _.id == appId }
> apps.getOrElse(throw new NotFoundException("unknown app: " + appId))
> }
> }
> {code}
> Although all application history infos are stored in a LinkedHashMap, here to
> code transforms the map to an iterator and then uses the find() api which is
> O( n) instead of O(1) from a map.get() operation.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
