[
https://issues.apache.org/jira/browse/SPARK-30073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16984782#comment-16984782
]
Jungtaek Lim commented on SPARK-30073:
--------------------------------------
Are you playing with master branch hence set the affected version as 3.0?
Because the stack trace is not in line with recent master, and I'm not seeing
explicit call of count().
If you encounter this with master branch, it would be ideal to include the
commit sha as well. If you're investigating this issue yourself, that would be
great.
> HistoryPage render "count" cost too much time
> ---------------------------------------------
>
> Key: SPARK-30073
> URL: https://issues.apache.org/jira/browse/SPARK-30073
> Project: Spark
> Issue Type: Improvement
> Components: Web UI
> Affects Versions: 3.0.0
> Reporter: deshanxiao
> Priority: Major
>
> {code:java}
> "qtp1010584177-537" #537 daemon prio=5 os_prio=0 tid=0x00007f2734185000
> nid=0x2c744 runnable [0x00007f23775e6000]
> java.lang.Thread.State: RUNNABLE
> at
> org.fusesource.leveldbjni.internal.NativeIterator$IteratorJNI.Prev(Native
> Method)
> at
> org.fusesource.leveldbjni.internal.NativeIterator.prev(NativeIterator.java:162)
> at
> org.fusesource.leveldbjni.internal.JniDBIterator.peekPrev(JniDBIterator.java:128)
> at
> org.fusesource.leveldbjni.internal.JniDBIterator.prev(JniDBIterator.java:144)
> at
> org.apache.spark.util.kvstore.LevelDBIterator.loadNext(LevelDBIterator.java:218)
> at
> org.apache.spark.util.kvstore.LevelDBIterator.hasNext(LevelDBIterator.java:111)
> at
> scala.collection.convert.Wrappers$JIteratorWrapper.hasNext(Wrappers.scala:42)
> at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:408)
> at scala.collection.Iterator$class.foreach(Iterator.scala:893)
> at scala.collection.AbstractIterator.foreach(Iterator.scala:1336)
> at
> scala.collection.TraversableOnce$class.count(TraversableOnce.scala:115)
> at scala.collection.AbstractIterator.count(Iterator.scala:1336)
> at
> org.apache.spark.deploy.history.HistoryPage.render(HistoryPage.scala:50)
> at org.apache.spark.ui.WebUI$$anonfun$2.apply(WebUI.scala:82)
> at org.apache.spark.ui.WebUI$$anonfun$2.apply(WebUI.scala:82)
> at org.apache.spark.ui.JettyUtils$$anon$3.doGet(JettyUtils.scala:90)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:687)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
> at
> org.spark_project.jetty.servlet.ServletHolder.handle(ServletHolder.java:848)
> at
> org.spark_project.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:584)
> at org.spark_project.jetty.server.handler.ContextHandler.do
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
