Re: Can't view executor logs in web UI on Windows
Hi, I am new to spark and trying to learn doing some programs on windows. I faced the same issue when running on windows. Cannot open Spark WebUI, I can see the output and output folder has the information that I needed but logs states that the WebUI is stopped. Does anyone have solution to view WebUI in windows? Thanks, Asmath On Tue, Feb 9, 2016 at 9:58 AM, Mark Pavey <mark.pa...@thefilter.com> wrote: > I have submitted a pull request: > https://github.com/apache/spark/pull/11135. > > Mark > > > -Original Message- > From: Mark Pavey [mailto:mark.pa...@thefilter.com] > Sent: 05 February 2016 17:09 > To: 'Ted Yu' > Cc: user@spark.apache.org > Subject: RE: Can't view executor logs in web UI on Windows > > We have created JIRA ticket > https://issues.apache.org/jira/browse/SPARK-13142 and will submit a pull > request next week. > > Mark > > > -Original Message- > From: Ted Yu [mailto:yuzhih...@gmail.com] > Sent: 01 February 2016 14:24 > To: Mark Pavey > Cc: user@spark.apache.org > Subject: Re: Can't view executor logs in web UI on Windows > > I did a brief search but didn't find relevant JIRA either. > > You can create a JIRA and submit pull request for the fix. > > Cheers > > > On Feb 1, 2016, at 5:13 AM, Mark Pavey <mark.pa...@thefilter.com> wrote: > > > > I am running Spark on Windows. When I try to view the Executor logs in > > the UI I get the following error: > > > > HTTP ERROR 500 > > > > Problem accessing /logPage/. Reason: > > > >Server Error > > Caused by: > > > > java.net.URISyntaxException: Illegal character in path at index 1: > > .\work/app-20160129154716-0038/2/ > >at java.net.URI$Parser.fail(Unknown Source) > >at java.net.URI$Parser.checkChars(Unknown Source) > >at java.net.URI$Parser.parseHierarchical(Unknown Source) > >at java.net.URI$Parser.parse(Unknown Source) > >at java.net.URI.(Unknown Source) > >at org.apache.spark.deploy.worker.ui.LogPage.getLog(LogPage.scala:141) > >at org.apache.spark.deploy.worker.ui.LogPage.render(LogPage.scala:78) > >at org.apache.spark.ui.WebUI$$anonfun$2.apply(WebUI.scala:79) > >at org.apache.spark.ui.WebUI$$anonfun$2.apply(WebUI.scala:79) > >at org.apache.spark.ui.JettyUtils$$anon$1.doGet(JettyUtils.scala:69) > >at javax.servlet.http.HttpServlet.service(HttpServlet.java:735) > >at javax.servlet.http.HttpServlet.service(HttpServlet.java:848) > >at > > > > org.spark-project.jetty.servlet.ServletHolder.handle(ServletHolder.java:684) > >at > > > > org.spark-project.jetty.servlet.ServletHandler.doHandle(ServletHandler.java: > 501) > >at > > > > org.spark-project.jetty.server.handler.ContextHandler.doHandle(ContextHandle > r.java:1086) > >at > > > > org.spark-project.jetty.servlet.ServletHandler.doScope(ServletHandler.java:4 > 28) > >at > > > > org.spark-project.jetty.server.handler.ContextHandler.doScope(ContextHandler > .java:1020) > >at > > > > org.spark-project.jetty.server.handler.ScopedHandler.handle(ScopedHandler.ja > va:135) > >at > > > > org.spark-project.jetty.server.handler.GzipHandler.handle(GzipHandler.java:2 > 64) > >at > > > > org.spark-project.jetty.server.handler.ContextHandlerCollection.handle(Conte > xtHandlerCollection.java:255) > >at > > > > org.spark-project.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper. > java:116) > >at org.spark-project.jetty.server.Server.handle(Server.java:370) > >at > > > > org.spark-project.jetty.server.AbstractHttpConnection.handleRequest(Abstract > HttpConnection.java:494) > >at > > > > org.spark-project.jetty.server.AbstractHttpConnection.headerComplete(Abstrac > tHttpConnection.java:971) > >at > > > > org.spark-project.jetty.server.AbstractHttpConnection$RequestHandler.headerC > omplete(AbstractHttpConnection.java:1033) > >at > org.spark-project.jetty.http.HttpParser.parseNext(HttpParser.java:644) > >at > > > org.spark-project.jetty.http.HttpParser.parseAvailable(HttpParser.java:235) > >at > > > > org.spark-project.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnectio > n.java:82) > >at > > > > org.spark-project.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEnd > Point.java:667) > >at > > > > org.spark-project.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndP > oint.java:52) > >at > > > > org.spark-project.jetty.util.thr
RE: Can't view executor logs in web UI on Windows
I have submitted a pull request: https://github.com/apache/spark/pull/11135. Mark -Original Message- From: Mark Pavey [mailto:mark.pa...@thefilter.com] Sent: 05 February 2016 17:09 To: 'Ted Yu' Cc: user@spark.apache.org Subject: RE: Can't view executor logs in web UI on Windows We have created JIRA ticket https://issues.apache.org/jira/browse/SPARK-13142 and will submit a pull request next week. Mark -Original Message- From: Ted Yu [mailto:yuzhih...@gmail.com] Sent: 01 February 2016 14:24 To: Mark Pavey Cc: user@spark.apache.org Subject: Re: Can't view executor logs in web UI on Windows I did a brief search but didn't find relevant JIRA either. You can create a JIRA and submit pull request for the fix. Cheers > On Feb 1, 2016, at 5:13 AM, Mark Pavey <mark.pa...@thefilter.com> wrote: > > I am running Spark on Windows. When I try to view the Executor logs in > the UI I get the following error: > > HTTP ERROR 500 > > Problem accessing /logPage/. Reason: > >Server Error > Caused by: > > java.net.URISyntaxException: Illegal character in path at index 1: > .\work/app-20160129154716-0038/2/ >at java.net.URI$Parser.fail(Unknown Source) >at java.net.URI$Parser.checkChars(Unknown Source) >at java.net.URI$Parser.parseHierarchical(Unknown Source) >at java.net.URI$Parser.parse(Unknown Source) >at java.net.URI.(Unknown Source) >at org.apache.spark.deploy.worker.ui.LogPage.getLog(LogPage.scala:141) >at org.apache.spark.deploy.worker.ui.LogPage.render(LogPage.scala:78) >at org.apache.spark.ui.WebUI$$anonfun$2.apply(WebUI.scala:79) >at org.apache.spark.ui.WebUI$$anonfun$2.apply(WebUI.scala:79) >at org.apache.spark.ui.JettyUtils$$anon$1.doGet(JettyUtils.scala:69) >at javax.servlet.http.HttpServlet.service(HttpServlet.java:735) >at javax.servlet.http.HttpServlet.service(HttpServlet.java:848) >at > org.spark-project.jetty.servlet.ServletHolder.handle(ServletHolder.java:684) >at > org.spark-project.jetty.servlet.ServletHandler.doHandle(ServletHandler.java: 501) >at > org.spark-project.jetty.server.handler.ContextHandler.doHandle(ContextHandle r.java:1086) >at > org.spark-project.jetty.servlet.ServletHandler.doScope(ServletHandler.java:4 28) >at > org.spark-project.jetty.server.handler.ContextHandler.doScope(ContextHandler .java:1020) >at > org.spark-project.jetty.server.handler.ScopedHandler.handle(ScopedHandler.ja va:135) >at > org.spark-project.jetty.server.handler.GzipHandler.handle(GzipHandler.java:2 64) >at > org.spark-project.jetty.server.handler.ContextHandlerCollection.handle(Conte xtHandlerCollection.java:255) >at > org.spark-project.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper. java:116) >at org.spark-project.jetty.server.Server.handle(Server.java:370) >at > org.spark-project.jetty.server.AbstractHttpConnection.handleRequest(Abstract HttpConnection.java:494) >at > org.spark-project.jetty.server.AbstractHttpConnection.headerComplete(Abstrac tHttpConnection.java:971) >at > org.spark-project.jetty.server.AbstractHttpConnection$RequestHandler.headerC omplete(AbstractHttpConnection.java:1033) >at org.spark-project.jetty.http.HttpParser.parseNext(HttpParser.java:644) >at > org.spark-project.jetty.http.HttpParser.parseAvailable(HttpParser.java:235) >at > org.spark-project.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnectio n.java:82) >at > org.spark-project.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEnd Point.java:667) >at > org.spark-project.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndP oint.java:52) >at > org.spark-project.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool .java:608) >at > org.spark-project.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool. java:543) >at java.lang.Thread.run(Unknown Source) > > > > Looking at the source code for > org.apache.spark.deploy.worker.ui.LogPage.getLog reveals the following: > - At line 141 the constructor of java.net.URI is called with the path > to the log directory as a String argument. This string > (".\work/app-20160129154716-0038/2/" in example above) contains a > backslash, which is an illegal character for the URI constructor. > - The component of the path containing the backslash is created at > line 71 by calling the getPath method on a java.io.File object. > Because it is running on Windows it uses the default Windows file > separator, which is a backslash. > > I am using Spark 1.5.1 but the source code appears unchanged in 1.6.0. > > I haven't been able to find an open issue for this but if there is one > could possibly subm
RE: Can't view executor logs in web UI on Windows
We have created JIRA ticket https://issues.apache.org/jira/browse/SPARK-13142 and will submit a pull request next week. Mark -Original Message- From: Ted Yu [mailto:yuzhih...@gmail.com] Sent: 01 February 2016 14:24 To: Mark Pavey Cc: user@spark.apache.org Subject: Re: Can't view executor logs in web UI on Windows I did a brief search but didn't find relevant JIRA either. You can create a JIRA and submit pull request for the fix. Cheers > On Feb 1, 2016, at 5:13 AM, Mark Pavey <mark.pa...@thefilter.com> wrote: > > I am running Spark on Windows. When I try to view the Executor logs in > the UI I get the following error: > > HTTP ERROR 500 > > Problem accessing /logPage/. Reason: > >Server Error > Caused by: > > java.net.URISyntaxException: Illegal character in path at index 1: > .\work/app-20160129154716-0038/2/ >at java.net.URI$Parser.fail(Unknown Source) >at java.net.URI$Parser.checkChars(Unknown Source) >at java.net.URI$Parser.parseHierarchical(Unknown Source) >at java.net.URI$Parser.parse(Unknown Source) >at java.net.URI.(Unknown Source) >at org.apache.spark.deploy.worker.ui.LogPage.getLog(LogPage.scala:141) >at org.apache.spark.deploy.worker.ui.LogPage.render(LogPage.scala:78) >at org.apache.spark.ui.WebUI$$anonfun$2.apply(WebUI.scala:79) >at org.apache.spark.ui.WebUI$$anonfun$2.apply(WebUI.scala:79) >at org.apache.spark.ui.JettyUtils$$anon$1.doGet(JettyUtils.scala:69) >at javax.servlet.http.HttpServlet.service(HttpServlet.java:735) >at javax.servlet.http.HttpServlet.service(HttpServlet.java:848) >at > org.spark-project.jetty.servlet.ServletHolder.handle(ServletHolder.java:684) >at > org.spark-project.jetty.servlet.ServletHandler.doHandle(ServletHandler.java: 501) >at > org.spark-project.jetty.server.handler.ContextHandler.doHandle(ContextHandle r.java:1086) >at > org.spark-project.jetty.servlet.ServletHandler.doScope(ServletHandler.java:4 28) >at > org.spark-project.jetty.server.handler.ContextHandler.doScope(ContextHandler .java:1020) >at > org.spark-project.jetty.server.handler.ScopedHandler.handle(ScopedHandler.ja va:135) >at > org.spark-project.jetty.server.handler.GzipHandler.handle(GzipHandler.java:2 64) >at > org.spark-project.jetty.server.handler.ContextHandlerCollection.handle(Conte xtHandlerCollection.java:255) >at > org.spark-project.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper. java:116) >at org.spark-project.jetty.server.Server.handle(Server.java:370) >at > org.spark-project.jetty.server.AbstractHttpConnection.handleRequest(Abstract HttpConnection.java:494) >at > org.spark-project.jetty.server.AbstractHttpConnection.headerComplete(Abstrac tHttpConnection.java:971) >at > org.spark-project.jetty.server.AbstractHttpConnection$RequestHandler.headerC omplete(AbstractHttpConnection.java:1033) >at org.spark-project.jetty.http.HttpParser.parseNext(HttpParser.java:644) >at > org.spark-project.jetty.http.HttpParser.parseAvailable(HttpParser.java:235) >at > org.spark-project.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnectio n.java:82) >at > org.spark-project.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEnd Point.java:667) >at > org.spark-project.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndP oint.java:52) >at > org.spark-project.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool .java:608) >at > org.spark-project.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool. java:543) >at java.lang.Thread.run(Unknown Source) > > > > Looking at the source code for > org.apache.spark.deploy.worker.ui.LogPage.getLog reveals the following: > - At line 141 the constructor of java.net.URI is called with the path > to the log directory as a String argument. This string > (".\work/app-20160129154716-0038/2/" in example above) contains a > backslash, which is an illegal character for the URI constructor. > - The component of the path containing the backslash is created at > line 71 by calling the getPath method on a java.io.File object. > Because it is running on Windows it uses the default Windows file > separator, which is a backslash. > > I am using Spark 1.5.1 but the source code appears unchanged in 1.6.0. > > I haven't been able to find an open issue for this but if there is one > could possibly submit a pull request for it. > > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Can-t-view-executo > r-logs-in-web-UI-on-Windows-tp26122.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > ---
Can't view executor logs in web UI on Windows
I am running Spark on Windows. When I try to view the Executor logs in the UI I get the following error: HTTP ERROR 500 Problem accessing /logPage/. Reason: Server Error Caused by: java.net.URISyntaxException: Illegal character in path at index 1: .\work/app-20160129154716-0038/2/ at java.net.URI$Parser.fail(Unknown Source) at java.net.URI$Parser.checkChars(Unknown Source) at java.net.URI$Parser.parseHierarchical(Unknown Source) at java.net.URI$Parser.parse(Unknown Source) at java.net.URI.(Unknown Source) at org.apache.spark.deploy.worker.ui.LogPage.getLog(LogPage.scala:141) at org.apache.spark.deploy.worker.ui.LogPage.render(LogPage.scala:78) at org.apache.spark.ui.WebUI$$anonfun$2.apply(WebUI.scala:79) at org.apache.spark.ui.WebUI$$anonfun$2.apply(WebUI.scala:79) at org.apache.spark.ui.JettyUtils$$anon$1.doGet(JettyUtils.scala:69) at javax.servlet.http.HttpServlet.service(HttpServlet.java:735) at javax.servlet.http.HttpServlet.service(HttpServlet.java:848) at org.spark-project.jetty.servlet.ServletHolder.handle(ServletHolder.java:684) at org.spark-project.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:501) at org.spark-project.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086) at org.spark-project.jetty.servlet.ServletHandler.doScope(ServletHandler.java:428) at org.spark-project.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020) at org.spark-project.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135) at org.spark-project.jetty.server.handler.GzipHandler.handle(GzipHandler.java:264) at org.spark-project.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255) at org.spark-project.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116) at org.spark-project.jetty.server.Server.handle(Server.java:370) at org.spark-project.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:494) at org.spark-project.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:971) at org.spark-project.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1033) at org.spark-project.jetty.http.HttpParser.parseNext(HttpParser.java:644) at org.spark-project.jetty.http.HttpParser.parseAvailable(HttpParser.java:235) at org.spark-project.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82) at org.spark-project.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:667) at org.spark-project.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52) at org.spark-project.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608) at org.spark-project.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) at java.lang.Thread.run(Unknown Source) Looking at the source code for org.apache.spark.deploy.worker.ui.LogPage.getLog reveals the following: - At line 141 the constructor of java.net.URI is called with the path to the log directory as a String argument. This string (".\work/app-20160129154716-0038/2/" in example above) contains a backslash, which is an illegal character for the URI constructor. - The component of the path containing the backslash is created at line 71 by calling the getPath method on a java.io.File object. Because it is running on Windows it uses the default Windows file separator, which is a backslash. I am using Spark 1.5.1 but the source code appears unchanged in 1.6.0. I haven't been able to find an open issue for this but if there is one could possibly submit a pull request for it. -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Can-t-view-executor-logs-in-web-UI-on-Windows-tp26122.html Sent from the Apache Spark User List mailing list archive at Nabble.com. - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
Re: Can't view executor logs in web UI on Windows
I did a brief search but didn't find relevant JIRA either. You can create a JIRA and submit pull request for the fix. Cheers > On Feb 1, 2016, at 5:13 AM, Mark Paveywrote: > > I am running Spark on Windows. When I try to view the Executor logs in the UI > I get the following error: > > HTTP ERROR 500 > > Problem accessing /logPage/. Reason: > >Server Error > Caused by: > > java.net.URISyntaxException: Illegal character in path at index 1: > .\work/app-20160129154716-0038/2/ >at java.net.URI$Parser.fail(Unknown Source) >at java.net.URI$Parser.checkChars(Unknown Source) >at java.net.URI$Parser.parseHierarchical(Unknown Source) >at java.net.URI$Parser.parse(Unknown Source) >at java.net.URI.(Unknown Source) >at org.apache.spark.deploy.worker.ui.LogPage.getLog(LogPage.scala:141) >at org.apache.spark.deploy.worker.ui.LogPage.render(LogPage.scala:78) >at org.apache.spark.ui.WebUI$$anonfun$2.apply(WebUI.scala:79) >at org.apache.spark.ui.WebUI$$anonfun$2.apply(WebUI.scala:79) >at org.apache.spark.ui.JettyUtils$$anon$1.doGet(JettyUtils.scala:69) >at javax.servlet.http.HttpServlet.service(HttpServlet.java:735) >at javax.servlet.http.HttpServlet.service(HttpServlet.java:848) >at > org.spark-project.jetty.servlet.ServletHolder.handle(ServletHolder.java:684) >at > org.spark-project.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:501) >at > org.spark-project.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086) >at > org.spark-project.jetty.servlet.ServletHandler.doScope(ServletHandler.java:428) >at > org.spark-project.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020) >at > org.spark-project.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135) >at > org.spark-project.jetty.server.handler.GzipHandler.handle(GzipHandler.java:264) >at > org.spark-project.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255) >at > org.spark-project.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116) >at org.spark-project.jetty.server.Server.handle(Server.java:370) >at > org.spark-project.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:494) >at > org.spark-project.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:971) >at > org.spark-project.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1033) >at org.spark-project.jetty.http.HttpParser.parseNext(HttpParser.java:644) >at > org.spark-project.jetty.http.HttpParser.parseAvailable(HttpParser.java:235) >at > org.spark-project.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82) >at > org.spark-project.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:667) >at > org.spark-project.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52) >at > org.spark-project.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608) >at > org.spark-project.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) >at java.lang.Thread.run(Unknown Source) > > > > Looking at the source code for > org.apache.spark.deploy.worker.ui.LogPage.getLog reveals the following: > - At line 141 the constructor of java.net.URI is called with the path to > the log directory as a String argument. This string > (".\work/app-20160129154716-0038/2/" in example above) contains a backslash, > which is an illegal character for the URI constructor. > - The component of the path containing the backslash is created at line 71 > by calling the getPath method on a java.io.File object. Because it is > running on Windows it uses the default Windows file separator, which is a > backslash. > > I am using Spark 1.5.1 but the source code appears unchanged in 1.6.0. > > I haven't been able to find an open issue for this but if there is one could > possibly submit a pull request for it. > > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Can-t-view-executor-logs-in-web-UI-on-Windows-tp26122.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > - > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > For additional commands, e-mail: user-h...@spark.apache.org > - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org