Hi Ted, You're absolutely right - twice! It was initially indeed in spark-shell and there can only be only active SparkContext. I did `sc.stop`, but it didn't help much.
That's why I switched to a Scala/sbt/Spark project and run it all from scratch (as shown). The stacktrace was showing up regularly. Give it a shot yourself as it's easy to reproduce - build Spark from the sources and have a project with libraryDependencies set with Spark core 1.6.0-SNAPSHOT. Pozdrawiam, Jacek -- Jacek Laskowski | http://blog.japila.pl | http://blog.jaceklaskowski.pl Follow me at https://twitter.com/jaceklaskowski Upvote at http://stackoverflow.com/users/1305344/jacek-laskowski On Sat, Oct 3, 2015 at 7:15 PM, Ted Yu <yuzhih...@gmail.com> wrote: > Did you use spark-shell ? > > In spark-shell, there can only be one running SparkContext which is created > automatically. > > Cheers > > On Sat, Oct 3, 2015 at 11:27 AM, Jacek Laskowski <ja...@japila.pl> wrote: >> >> Hi, >> >> The following WARN happens in Spark built from today's sources. There >> were some changes about RPC so it may be related. Should I report an >> issue in JIRA? >> >> I use a sbt project and `sbt console` to play with Spark. >> >> ``` >> scala> import org.apache.spark.SparkConf >> import org.apache.spark.SparkConf >> >> scala> val conf = new >> SparkConf().setMaster("local[*]").setAppName("Hello world!") >> conf: org.apache.spark.SparkConf = org.apache.spark.SparkConf@7a8f69d6 >> >> scala> import org.apache.spark.SparkContext >> import org.apache.spark.SparkContext >> >> scala> val sc = new SparkContext(conf) >> ``` >> >> So far so good. After a while I can see my CPU going up and the REPL >> shows up with the WARN: >> >> ``` >> 15/10/03 14:26:03 WARN NettyRpcEndpointRef: Error sending message >> [message = >> Heartbeat(driver,[Lscala.Tuple2;@711362a6,BlockManagerId(driver, >> localhost, 50804))] in 1 attempts >> org.apache.spark.rpc.RpcTimeoutException: Futures timed out after [120 >> seconds]. This timeout is controlled by spark.rpc.askTimeout >> at >> org.apache.spark.rpc.RpcTimeout.org$apache$spark$rpc$RpcTimeout$$createRpcTimeoutException(RpcEnv.scala:215) >> at >> org.apache.spark.rpc.RpcTimeout$$anonfun$addMessageIfTimeout$1.applyOrElse(RpcEnv.scala:230) >> at >> org.apache.spark.rpc.RpcTimeout$$anonfun$addMessageIfTimeout$1.applyOrElse(RpcEnv.scala:226) >> at >> scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:36) >> at org.apache.spark.rpc.RpcTimeout.awaitResult(RpcEnv.scala:243) >> at >> org.apache.spark.rpc.RpcEndpointRef.askWithRetry(RpcEndpointRef.scala:101) >> at >> org.apache.spark.rpc.RpcEndpointRef.askWithRetry(RpcEndpointRef.scala:77) >> at >> org.apache.spark.executor.Executor.org$apache$spark$executor$Executor$$reportHeartBeat(Executor.scala:452) >> at >> org.apache.spark.executor.Executor$$anon$1$$anonfun$run$1.apply$mcV$sp(Executor.scala:472) >> at >> org.apache.spark.executor.Executor$$anon$1$$anonfun$run$1.apply(Executor.scala:472) >> at >> org.apache.spark.executor.Executor$$anon$1$$anonfun$run$1.apply(Executor.scala:472) >> at org.apache.spark.util.Utils$.logUncaughtExceptions(Utils.scala:1697) >> at org.apache.spark.executor.Executor$$anon$1.run(Executor.scala:472) >> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) >> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) >> at >> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) >> at >> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) >> at java.lang.Thread.run(Thread.java:745) >> Caused by: java.util.concurrent.TimeoutException: Futures timed out >> after [120 seconds] >> at scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:219) >> at scala.concurrent.impl.Promise$DefaultPromise.result(Promise.scala:223) >> at scala.concurrent.Await$$anonfun$result$1.apply(package.scala:190) >> at >> scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:53) >> at scala.concurrent.Await$.result(package.scala:190) >> at org.apache.spark.rpc.RpcTimeout.awaitResult(RpcEnv.scala:242) >> ... 15 more >> ``` >> >> Pozdrawiam, >> Jacek >> >> -- >> Jacek Laskowski | http://blog.japila.pl | http://blog.jaceklaskowski.pl >> Follow me at https://twitter.com/jaceklaskowski >> Upvote at http://stackoverflow.com/users/1305344/jacek-laskowski >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org >> For additional commands, e-mail: user-h...@spark.apache.org >> > --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org