Still, it would be good to know what happened exactly. Why did the netty dependency expect Java 8? Did you build your app on a machine with Java 8 and deploy on a Java 7 machine?
Anyway, I played with the 1.6.0 spark-shell using Java 7 and it worked fine. I also looked at the distribution's class files using e.g., $ cd $HOME/spark/spark-1.6.0-bin-hadoop2.6 $ jar xf lib/spark-assembly-1.6.0-hadoop2.6.0.jar org/apache/spark/rpc/netty/Dispatcher.class $ javap -classpath . -verbose org.apache.spark.rpc.netty.Dispatcher | grep version minor version: 0 major version: 50 So, it was compiled with Java 6 (see https://en.wikipedia.org/wiki/Java_class_file). So, it doesn't appear to be a Spark build issue. dean Dean Wampler, Ph.D. Author: Programming Scala, 2nd Edition <http://shop.oreilly.com/product/0636920033073.do> (O'Reilly) Typesafe <http://typesafe.com> @deanwampler <http://twitter.com/deanwampler> http://polyglotprogramming.com On Tue, Jan 5, 2016 at 9:01 AM, Yiannis Gkoufas <johngou...@gmail.com> wrote: > Hi Dean, > > thanks so much for the response! It works without a problem now! > > On 5 January 2016 at 14:33, Dean Wampler <deanwamp...@gmail.com> wrote: > >> ConcurrentHashMap.keySet() returning a KeySetView is a Java 8 method. The >> Java 7 method returns a Set. Are you running Java 7? What happens if you >> run Java 8? >> >> Dean Wampler, Ph.D. >> Author: Programming Scala, 2nd Edition >> <http://shop.oreilly.com/product/0636920033073.do> (O'Reilly) >> Typesafe <http://typesafe.com> >> @deanwampler <http://twitter.com/deanwampler> >> http://polyglotprogramming.com >> >> On Tue, Jan 5, 2016 at 8:29 AM, Yiannis Gkoufas <johngou...@gmail.com> >> wrote: >> >>> Hi there, >>> >>> I have been using Spark 1.5.2 on my cluster without a problem and wanted >>> to try Spark 1.6.0. >>> I have the exact same configuration on both clusters. >>> I am able to start the Standalone Cluster but I fail to submit a job >>> getting errors like the following: >>> >>> 16/01/05 14:24:14 INFO AppClient$ClientEndpoint: Connecting to master >>> spark://my-ip:7077... >>> 16/01/05 14:24:34 INFO AppClient$ClientEndpoint: Connecting to master >>> spark://my-ip:7077... >>> 16/01/05 14:24:54 INFO AppClient$ClientEndpoint: Connecting to master >>> spark://my-ip:7077... >>> 16/01/05 14:24:54 INFO AppClient$ClientEndpoint: Connecting to master >>> spark://my-ip:7077... >>> 16/01/05 14:24:54 WARN TransportChannelHandler: Exception in connection >>> from my-ip/X.XXX.XX.XX:7077 >>> java.lang.NoSuchMethodError: >>> java.util.concurrent.ConcurrentHashMap.keySet()Ljava/util/concurrent/ConcurrentHashMap$KeySetView; >>> at org.apache.spark.rpc.netty.Dispatcher.postToAll(Dispatcher.scala:106) >>> at >>> org.apache.spark.rpc.netty.NettyRpcHandler.internalReceive(NettyRpcEnv.scala:586) >>> at >>> org.apache.spark.rpc.netty.NettyRpcHandler.receive(NettyRpcEnv.scala:577) >>> at >>> org.apache.spark.network.server.TransportRequestHandler.processOneWayMessage(TransportRequestHandler.java:170) >>> at >>> org.apache.spark.network.server.TransportRequestHandler.handle(TransportRequestHandler.java:104) >>> at >>> org.apache.spark.network.server.TransportChannelHandler.channelRead0(TransportChannelHandler.java:104) >>> at >>> org.apache.spark.network.server.TransportChannelHandler.channelRead0(TransportChannelHandler.java:51) >>> at >>> io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105) >>> at >>> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:308) >>> at >>> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:294) >>> at >>> io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:266) >>> at >>> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:308) >>> at >>> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:294) >>> at >>> io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) >>> at >>> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:308) >>> at >>> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:294) >>> at >>> org.apache.spark.network.util.TransportFrameDecoder.channelRead(TransportFrameDecoder.java:86) >>> at >>> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:308) >>> at >>> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:294) >>> at >>> io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:846) >>> at >>> io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:131) >>> at >>> io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511) >>> at >>> io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468) >>> at >>> io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382) >>> at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354) >>> at >>> io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111) >>> at java.lang.Thread.run(Thread.java:745) >>> >>> Has anyone else had similar problems? >>> >>> Thanks a lot >>> >>> >> >
