GitHub user scwf opened a pull request:
https://github.com/apache/spark/pull/312
method getAllPools in SC throws NPE
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/home/wangfei/spark0403/spark-master/examples/target/scala-2.10/spark-examples_2.10-assembly-1.0.0-SNAPSHOT.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/home/wangfei/spark0403/spark-master/assembly/target/scala-2.10/spark-assembly_2.10-1.0.0-SNAPSHOT-hadoop2.3.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
14/04/03 16:43:19 INFO SecurityManager: SecurityManager, is authentication
enabled: false are ui acls enabled: false users with view permissions: Set(root)
14/04/03 16:43:19 INFO Slf4jLogger: Slf4jLogger started
14/04/03 16:43:19 INFO Remoting: Starting remoting
14/04/03 16:43:20 INFO Remoting: Remoting started; listening on addresses
:[akka.tcp://spark@VM-13:55231]
14/04/03 16:43:20 INFO Remoting: Remoting now listens on addresses:
[akka.tcp://spark@VM-13:55231]
14/04/03 16:43:20 INFO SparkEnv: Registering BlockManagerMaster
14/04/03 16:43:20 INFO DiskBlockManager: Created local directory at
/tmp/spark-local-20140403164320-b637
14/04/03 16:43:20 INFO MemoryStore: MemoryStore started with capacity
1141.5 MB.
14/04/03 16:43:20 INFO ConnectionManager: Bound socket to port 33813 with
id = ConnectionManagerId(VM-13,33813)
14/04/03 16:43:20 INFO BlockManagerMaster: Trying to register BlockManager
14/04/03 16:43:20 INFO BlockManagerInfo: Registering block manager
VM-13:33813 with 1141.5 MB RAM
14/04/03 16:43:20 INFO BlockManagerMaster: Registered BlockManager
14/04/03 16:43:20 INFO HttpServer: Starting HTTP Server
14/04/03 16:43:21 INFO HttpBroadcast: Broadcast server started at
http://9.91.11.32:58746
14/04/03 16:43:21 INFO SparkEnv: Registering MapOutputTracker
14/04/03 16:43:21 INFO HttpFileServer: HTTP File server directory is
/tmp/spark-34b290f1-ca2f-489e-93c5-1c3e860874b0
14/04/03 16:43:21 INFO HttpServer: Starting HTTP Server
14/04/03 16:43:22 INFO SparkUI: Started Spark Web UI at http://VM-13:4040
14/04/03 16:43:22 WARN ServletHandler: /stages/
java.lang.NullPointerException
at org.apache.spark.SparkContext.getAllPools(SparkContext.scala:712)
at org.apache.spark.ui.jobs.IndexPage.render(IndexPage.scala:50)
at
org.apache.spark.ui.jobs.JobProgressUI$$anonfun$getHandlers$3.apply(JobProgressUI.scala:58)
at
org.apache.spark.ui.jobs.JobProgressUI$$anonfun$getHandlers$3.apply(JobProgressUI.scala:58)
at org.apache.spark.ui.JettyUtils$$anon$1.doGet(JettyUtils.scala:68)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
at
org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:684)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:501)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086)
at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:428)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
at
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
at org.eclipse.jetty.server.Server.handle(Server.java:370)
at
org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:494)
at
org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:971)
at
org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1033)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:644)
at
org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
at
org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
at
org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:667)
at
org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
at java.lang.Thread.run(Thread.java:662)
this is because rootPool method is not implemented in TaskSchedulerImpl
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/scwf/spark patch-1
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/312.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #312
----
commit 832741124a69a9f3993921f415cb3c93c52de001
Author: wangfei <[email protected]>
Date: 2014-04-03T09:24:09Z
method getAllPools in SC throws NPE
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/home/wangfei/spark0403/spark-master/examples/target/scala-2.10/spark-examples_2.10-assembly-1.0.0-SNAPSHOT.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/home/wangfei/spark0403/spark-master/assembly/target/scala-2.10/spark-assembly_2.10-1.0.0-SNAPSHOT-hadoop2.3.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
14/04/03 16:43:19 INFO SecurityManager: SecurityManager, is authentication
enabled: false are ui acls enabled: false users with view permissions: Set(root)
14/04/03 16:43:19 INFO Slf4jLogger: Slf4jLogger started
14/04/03 16:43:19 INFO Remoting: Starting remoting
14/04/03 16:43:20 INFO Remoting: Remoting started; listening on addresses
:[akka.tcp://spark@VM-13:55231]
14/04/03 16:43:20 INFO Remoting: Remoting now listens on addresses:
[akka.tcp://spark@VM-13:55231]
14/04/03 16:43:20 INFO SparkEnv: Registering BlockManagerMaster
14/04/03 16:43:20 INFO DiskBlockManager: Created local directory at
/tmp/spark-local-20140403164320-b637
14/04/03 16:43:20 INFO MemoryStore: MemoryStore started with capacity
1141.5 MB.
14/04/03 16:43:20 INFO ConnectionManager: Bound socket to port 33813 with
id = ConnectionManagerId(VM-13,33813)
14/04/03 16:43:20 INFO BlockManagerMaster: Trying to register BlockManager
14/04/03 16:43:20 INFO BlockManagerInfo: Registering block manager
VM-13:33813 with 1141.5 MB RAM
14/04/03 16:43:20 INFO BlockManagerMaster: Registered BlockManager
14/04/03 16:43:20 INFO HttpServer: Starting HTTP Server
14/04/03 16:43:21 INFO HttpBroadcast: Broadcast server started at
http://9.91.11.32:58746
14/04/03 16:43:21 INFO SparkEnv: Registering MapOutputTracker
14/04/03 16:43:21 INFO HttpFileServer: HTTP File server directory is
/tmp/spark-34b290f1-ca2f-489e-93c5-1c3e860874b0
14/04/03 16:43:21 INFO HttpServer: Starting HTTP Server
14/04/03 16:43:22 INFO SparkUI: Started Spark Web UI at http://VM-13:4040
14/04/03 16:43:22 WARN ServletHandler: /stages/
java.lang.NullPointerException
at org.apache.spark.SparkContext.getAllPools(SparkContext.scala:712)
at org.apache.spark.ui.jobs.IndexPage.render(IndexPage.scala:50)
at
org.apache.spark.ui.jobs.JobProgressUI$$anonfun$getHandlers$3.apply(JobProgressUI.scala:58)
at
org.apache.spark.ui.jobs.JobProgressUI$$anonfun$getHandlers$3.apply(JobProgressUI.scala:58)
at org.apache.spark.ui.JettyUtils$$anon$1.doGet(JettyUtils.scala:68)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
at
org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:684)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:501)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086)
at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:428)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
at
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
at org.eclipse.jetty.server.Server.handle(Server.java:370)
at
org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:494)
at
org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:971)
at
org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1033)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:644)
at
org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
at
org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
at
org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:667)
at
org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
at java.lang.Thread.run(Thread.java:662)
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
