I also have some issues with Spark 1.6.0
Attached stack trace below:
16/01/06 01:50:29 WARN cluster.SparkDeploySchedulerBackend: Application ID
is not initialized yet.
16/01/06 01:50:29 ERROR cluster.SparkDeploySchedulerBackend: Application
has been killed. Reason: All masters are unresponsive! Giving up.
16/01/06 01:50:29 INFO util.Utils: Successfully started service
'org.apache.spark.network.netty.NettyBlockTransferService' on port 32902.
16/01/06 01:50:29 INFO netty.NettyBlockTransferService: Server created on
32902
16/01/06 01:50:29 INFO storage.BlockManagerMaster: Trying to register
BlockManager
16/01/06 01:50:29 INFO storage.BlockManagerMasterEndpoint: Registering
block manager 172.16.187.61:32902 with 1140.4 MB RAM,
BlockManagerId(driver, 172.16.187.61, 32902)
16/01/06 01:50:29 INFO storage.BlockManagerMaster: Registered BlockManager
16/01/06 01:50:29 INFO handler.ContextHandler: stopped
o.s.j.s.ServletContextHandler{/stages/stage/kill,null}
16/01/06 01:50:29 INFO handler.ContextHandler: stopped
o.s.j.s.ServletContextHandler{/api,null}
16/01/06 01:50:29 INFO handler.ContextHandler: stopped
o.s.j.s.ServletContextHandler{/,null}
16/01/06 01:50:29 INFO handler.ContextHandler: stopped
o.s.j.s.ServletContextHandler{/static,null}
16/01/06 01:50:29 INFO handler.ContextHandler: stopped
o.s.j.s.ServletContextHandler{/executors/threadDump/json,null}
16/01/06 01:50:29 INFO handler.ContextHandler: stopped
o.s.j.s.ServletContextHandler{/executors/threadDump,null}
16/01/06 01:50:29 INFO handler.ContextHandler: stopped
o.s.j.s.ServletContextHandler{/executors/json,null}
16/01/06 01:50:29 INFO handler.ContextHandler: stopped
o.s.j.s.ServletContextHandler{/executors,null}
16/01/06 01:50:29 INFO handler.ContextHandler: stopped
o.s.j.s.ServletContextHandler{/environment/json,null}
16/01/06 01:50:29 INFO handler.ContextHandler: stopped
o.s.j.s.ServletContextHandler{/environment,null}
16/01/06 01:50:29 INFO handler.ContextHandler: stopped
o.s.j.s.ServletContextHandler{/storage/rdd/json,null}
16/01/06 01:50:29 INFO handler.ContextHandler: stopped
o.s.j.s.ServletContextHandler{/storage/rdd,null}
16/01/06 01:50:29 INFO handler.ContextHandler: stopped
o.s.j.s.ServletContextHandler{/storage/json,null}
16/01/06 01:50:29 INFO handler.ContextHandler: stopped
o.s.j.s.ServletContextHandler{/storage,null}
16/01/06 01:50:29 INFO handler.ContextHandler: stopped
o.s.j.s.ServletContextHandler{/stages/pool/json,null}
16/01/06 01:50:29 INFO handler.ContextHandler: stopped
o.s.j.s.ServletContextHandler{/stages/pool,null}
16/01/06 01:50:29 INFO handler.ContextHandler: stopped
o.s.j.s.ServletContextHandler{/stages/stage/json,null}
16/01/06 01:50:29 INFO handler.ContextHandler: stopped
o.s.j.s.ServletContextHandler{/stages/stage,null}
16/01/06 01:50:29 INFO handler.ContextHandler: stopped
o.s.j.s.ServletContextHandler{/stages/json,null}
16/01/06 01:50:29 INFO handler.ContextHandler: stopped
o.s.j.s.ServletContextHandler{/stages,null}
16/01/06 01:50:29 INFO handler.ContextHandler: stopped
o.s.j.s.ServletContextHandler{/jobs/job/json,null}
16/01/06 01:50:29 INFO handler.ContextHandler: stopped
o.s.j.s.ServletContextHandler{/jobs/job,null}
16/01/06 01:50:29 INFO handler.ContextHandler: stopped
o.s.j.s.ServletContextHandler{/jobs/json,null}
16/01/06 01:50:29 INFO handler.ContextHandler: stopped
o.s.j.s.ServletContextHandler{/jobs,null}
16/01/06 01:50:29 INFO ui.SparkUI: Stopped Spark web UI at
http://172.16.187.61:4041
16/01/06 01:50:29 INFO cluster.SparkDeploySchedulerBackend: Shutting down
all executors
16/01/06 01:50:29 INFO cluster.SparkDeploySchedulerBackend: Asking each
executor to shut down
16/01/06 01:50:29 WARN client.AppClient$ClientEndpoint: Drop
UnregisterApplication(null) because has not yet connected to master
16/01/06 01:50:29 ERROR spark.MapOutputTrackerMaster: Error communicating
with MapOutputTracker
java.lang.InterruptedException
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1325)
at scala.concurrent.impl.Promise$DefaultPromise.tryAwait(Promise.scala:208)
at scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:218)
at scala.concurrent.impl.Promise$DefaultPromise.result(Promise.scala:223)
at scala.concurrent.Await$$anonfun$result$1.apply(package.scala:107)
at
scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:53)
at scala.concurrent.Await$.result(package.scala:107)
at org.apache.spark.rpc.RpcTimeout.awaitResult(RpcTimeout.scala:75)
at
org.apache.spark.rpc.RpcEndpointRef.askWithRetry(RpcEndpointRef.scala:101)
at org.apache.spark.rpc.RpcEndpointRef.askWithRetry(RpcEndpointRef.scala:77)
at org.apache.spark.MapOutputTracker.askTracker(MapOutputTracker.scala:110)
at org.apache.spark.MapOutputTracker.sendTracker(MapOutputTracker.scala:120)
at org.apache.spark.MapOutputTrackerMaster.stop(MapOutputTracker.scala:462)
at org.apache.spark.SparkEnv.stop(SparkEnv.scala:93)
at
org.apache.spark.SparkContext$$anonfun$stop$12.apply$mcV$sp(SparkContext.scala:1756)
at org.apache.spark.util.Utils$.tryLogNonFatalError(Utils.scala:1229)
at org.apache.spark.SparkContext.stop(SparkContext.scala:1755)
at
org.apache.spark.scheduler.cluster.SparkDeploySchedulerBackend.dead(SparkDeploySchedulerBackend.scala:127)
at
org.apache.spark.deploy.client.AppClient$ClientEndpoint.markDead(AppClient.scala:264)
at
org.apache.spark.deploy.client.AppClient$ClientEndpoint$$anon$2$$anonfun$run$1.apply$mcV$sp(AppClient.scala:134)
at org.apache.spark.util.Utils$.tryOrExit(Utils.scala:1163)
at
org.apache.spark.deploy.client.AppClient$ClientEndpoint$$anon$2.run(AppClient.scala:129)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
16/01/06 01:50:29 INFO spark.MapOutputTrackerMasterEndpoint:
MapOutputTrackerMasterEndpoint stopped!
16/01/06 01:50:29 ERROR util.Utils: Uncaught exception in thread
appclient-registration-retry-thread
org.apache.spark.SparkException: Error communicating with MapOutputTracker
at org.apache.spark.MapOutputTracker.askTracker(MapOutputTracker.scala:114)
at org.apache.spark.MapOutputTracker.sendTracker(MapOutputTracker.scala:120)
at org.apache.spark.MapOutputTrackerMaster.stop(MapOutputTracker.scala:462)
at org.apache.spark.SparkEnv.stop(SparkEnv.scala:93)
at
org.apache.spark.SparkContext$$anonfun$stop$12.apply$mcV$sp(SparkContext.scala:1756)
at org.apache.spark.util.Utils$.tryLogNonFatalError(Utils.scala:1229)
at org.apache.spark.SparkContext.stop(SparkContext.scala:1755)
at
org.apache.spark.scheduler.cluster.SparkDeploySchedulerBackend.dead(SparkDeploySchedulerBackend.scala:127)
at
org.apache.spark.deploy.client.AppClient$ClientEndpoint.markDead(AppClient.scala:264)
at
org.apache.spark.deploy.client.AppClient$ClientEndpoint$$anon$2$$anonfun$run$1.apply$mcV$sp(AppClient.scala:134)
at org.apache.spark.util.Utils$.tryOrExit(Utils.scala:1163)
at
org.apache.spark.deploy.client.AppClient$ClientEndpoint$$anon$2.run(AppClient.scala:129)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.InterruptedException
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1325)
at scala.concurrent.impl.Promise$DefaultPromise.tryAwait(Promise.scala:208)
at scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:218)
at scala.concurrent.impl.Promise$DefaultPromise.result(Promise.scala:223)
at scala.concurrent.Await$$anonfun$result$1.apply(package.scala:107)
at
scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:53)
at scala.concurrent.Await$.result(package.scala:107)
at org.apache.spark.rpc.RpcTimeout.awaitResult(RpcTimeout.scala:75)
at
org.apache.spark.rpc.RpcEndpointRef.askWithRetry(RpcEndpointRef.scala:101)
at org.apache.spark.rpc.RpcEndpointRef.askWithRetry(RpcEndpointRef.scala:77)
at org.apache.spark.MapOutputTracker.askTracker(MapOutputTracker.scala:110)
... 18 more
16/01/06 01:50:29 INFO spark.SparkContext: Successfully stopped SparkContext
2016년 1월 6일 (수) 오전 4:53, Amos B. Elberg <amos.elb...@me.com>님이 작성:
> Moon - I don’t believe that PR actually works.
>
> From: moon soo Lee <m...@apache.org> <m...@apache.org>
> Reply: users@zeppelin.incubator.apache.org
> <users@zeppelin.incubator.apache.org>
> <users@zeppelin.incubator.apache.org>
> Date: January 5, 2016 at 2:51:14 PM
> To: users@zeppelin.incubator.apache.org
> <users@zeppelin.incubator.apache.org>
> <users@zeppelin.incubator.apache.org>
> Subject: Re: Spark 1.6.0?
>
> Hi Boris,
>
> There is pullrequest that supports Spark 1.6.0
> https://github.com/apache/incubator-zeppelin/pull/463. It's not merged
> yet. You may need to apply manually until it is get merged into master.
>
> Thanks,
> moon
>
> On Tue, Jan 5, 2016 at 11:38 AM Boris Schminke <schmink...@gmail.com>
> wrote:
>
>> Hi,
>> why can't I use Zeppelin with Spark 1.6.0?
>> Probably I could do it in developers' 0.6 version compiled from sources,
>> couldn't I?
>>
>> Regards,
>> Boris
>>
>
