My application logs remain stored as .inprogress files, e.g. "app-20150113190025-0004.inprogress” even after completion, could this have something to do with what is going on.
@ Ted Yu Where do I find the master log? It’s not very obviously labeled in my /tmp/ directory. Sorry if I should know, I’ve read through the documentation many times, but probably missed it. Do I need to set this in my configuration somehow? Could the problem actually be that I’m just not setting a conf property properly somewhere? @Robin East All of these work fine, the only issue I’m having is viewing the application ui after I submit a job to run. Further, the application runs and will even compute some of the outputs I want (they get stored to s3 and I’ve used the outputs in other models I’m making). It’s simply when I access the master web ui, click the application that is currently running, and then click “Application Detail UI” that I receive this error. When I an application has completed and I click the same link I receive: “Application history not found (app-201501131190025-004)" > On Jan 13, 2015, at 8:07 AM, Ted Yu <yuzhih...@gmail.com> wrote: > > Gabon: > Can you check the master log to see if there is some clue ? > > Cheers > > > > On Jan 13, 2015, at 2:03 AM, Robin East <robin.e...@xense.co.uk > <mailto:robin.e...@xense.co.uk>> wrote: > >> I’ve just pulled down the latest commits from github, and done the following: >> >> 1) >> mvn clean package -DskipTests >> >> builds fine >> >> 2) >> ./bin/spark-shell works >> >> 3) >> run SparkPi example with no problems: >> >> ./bin/run-example SparkPi 10 >> >> 4) >> Started a master >> >> ./sbin/start-master.sh >> >> grabbed the MasterWebUI from the master log - Started MasterWebUI at >> http://x.x.x.x:8080 <http://x.x.x.x:8080/> >> >> Can view the MasterWebUI from local browser >> >> 5) >> grabbed the spark url from the master log and started a local slave: >> >> ./bin/spark-class org.apache.spark.deploy.worker.Worker >> spark://<hostname>:7077 & >> >> 6) >> Ran jps to confirm both Master and Worker processes are present. >> >> 7) >> Ran SparkPi on the mini-cluster: >> >> MASTER=spark://<host>:7077 ./bin/run-example SparkPi 10 >> >> All worked fine, can see information in the MasterWebUI >> >> Which of these stops doesn’t work for you? I presume you’ve tried re-pulling >> from git and a clean build again. >> >> Robin >> On 13 Jan 2015, at 08:07, Ganon Pierce <ganon.pie...@me.com >> <mailto:ganon.pie...@me.com>> wrote: >> >>> After clean build still receiving the same error. >>> >>> >>> >>> On Jan 6, 2015, at 3:59 PM, Sean Owen <so...@cloudera.com >>> <mailto:so...@cloudera.com>> wrote: >>> >>>> FWIW I do not see any such error, after a "mvn -DskipTests clean package" >>>> and "./bin/spark-shell" from master. Maybe double-check you have done a >>>> full clean build. >>>> >>>> On Tue, Jan 6, 2015 at 9:09 PM, Ganon Pierce <ganon.pie...@me.com >>>> <mailto:ganon.pie...@me.com>> wrote: >>>> I’m attempting to build from the latest commit on git and receive the >>>> following error upon attempting to access the application web ui: >>>> >>>> HTTP ERROR: 500 >>>> >>>> Problem accessing /jobs/. Reason: >>>> >>>> Server Error >>>> Powered by Jetty:// >>>> >>>> My driver also prints this error: >>>> >>>> java.lang.UnsupportedOperationException: empty.max >>>> at scala.collection.TraversableOnce$class.max(TraversableOnce.scala:216) >>>> at scala.collection.AbstractTraversable.max(Traversable.scala:105) >>>> at org.apache.spark.ui.jobs.AllJobsPage.org >>>> <http://org.apache.spark.ui.jobs.alljobspage.org/>$apache$spark$ui$jobs$AllJobsPage$$makeRow$1(AllJobsPage.scala:46) >>>> at >>>> org.apache.spark.ui.jobs.AllJobsPage$$anonfun$jobsTable$1.apply(AllJobsPage.scala:91) >>>> at >>>> org.apache.spark.ui.jobs.AllJobsPage$$anonfun$jobsTable$1.apply(AllJobsPage.scala:91) >>>> at >>>> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244) >>>> at >>>> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244) >>>> at scala.collection.immutable.List.foreach(List.scala:318) >>>> at scala.collection.TraversableLike$class.map(TraversableLike.scala:244) >>>> at scala.collection.AbstractTraversable.map(Traversable.scala:105) >>>> at org.apache.spark.ui.jobs.AllJobsPage.jobsTable(AllJobsPage.scala:91) >>>> at org.apache.spark.ui.jobs.AllJobsPage.render(AllJobsPage.scala:106) >>>> at org.apache.spark.ui.WebUI$$anonfun$attachPage$1.apply(WebUI.scala:68) >>>> at org.apache.spark.ui.WebUI$$anonfun$attachPage$1.apply(WebUI.scala:68) >>>> at org.apache.spark.ui.JettyUtils$$anon$1.doGet(JettyUtils.scala:68) >>>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:735) >>>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:848) >>>> at >>>> org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:684) >>>> at >>>> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:501) >>>> at >>>> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086) >>>> at >>>> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:428) >>>> at >>>> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020) >>>> at >>>> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135) >>>> at >>>> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255) >>>> at >>>> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116) >>>> at org.eclipse.jetty.server.Server.handle(Server.java:370) >>>> at >>>> org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:494) >>>> at >>>> org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:971) >>>> at >>>> org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1033) >>>> at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:644) >>>> at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235) >>>> at >>>> org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82) >>>> at >>>> org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:667) >>>> at >>>> org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52) >>>> at >>>> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608) >>>> at >>>> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) >>>> at java.lang.Thread.run(Thread.java:745) >>>> >>>> >>>> Has the ui been disabled intentionally for development purposes, have I >>>> not set something up correctly, or is this a bug? >>>> >>
