[jira] [Commented] (SPARK-21348) Spark history Server load too slow when eventlog is large
[ https://issues.apache.org/jira/browse/SPARK-21348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16079421#comment-16079421 ] duyanghao commented on SPARK-21348: --- @Saisai Shao ok, i will follow [SPARK-18085](https://issues.apache.org/jira/browse/SPARK-18085) to trace this issue. > Spark history Server load too slow when eventlog is large > - > > Key: SPARK-21348 > URL: https://issues.apache.org/jira/browse/SPARK-21348 > Project: Spark > Issue Type: Bug > Components: Web UI >Affects Versions: 2.1.0 >Reporter: duyanghao >Priority: Critical > > There is a eventlog file in hdfs as below: > ``` > -rwxrwx--- 3 root supergroup 9634238709 2017-07-08 04:37 > hdfs://x/shared/spark-logs/x/.lz4 > ``` > and i start a spark history server to loading that eventlog file,but it is > too slow to get spark web ui through that spark history server. > it has requested for several hours but nothing about spark application web ui > appears in the browser > And the cpu usage of spark history server is 105.3% > total used-memory in host is 9G > The host has 48 Cores+130G Memory,enough Computing resources for spark > history server. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-21348) Spark history Server load too slow when eventlog is large
[ https://issues.apache.org/jira/browse/SPARK-21348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16079381#comment-16079381 ] Saisai Shao commented on SPARK-21348: - SPARK-18085 is trying to address this issue. [~duyanghao] are you trying to address this issue or you just report an issue ? If the latter I think you can close this JIRA and follow SPARK-18085. > Spark history Server load too slow when eventlog is large > - > > Key: SPARK-21348 > URL: https://issues.apache.org/jira/browse/SPARK-21348 > Project: Spark > Issue Type: Bug > Components: Web UI >Affects Versions: 2.1.0 >Reporter: duyanghao >Priority: Critical > > There is a eventlog file in hdfs as below: > ``` > -rwxrwx--- 3 root supergroup 9634238709 2017-07-08 04:37 > hdfs://x/shared/spark-logs/x/.lz4 > ``` > and i start a spark history server to loading that eventlog file,but it is > too slow to get spark web ui through that spark history server. > it has requested for several hours but nothing about spark application web ui > appears in the browser > And the cpu usage of spark history server is 105.3% > total used-memory in host is 9G > The host has 48 Cores+130G Memory,enough Computing resources for spark > history server. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
