This indicates your app is not actually using the version of the HDFS client you think. You built Spark from source with the right deps it seems, but are you sure you linked to your build in your app?
On Fri, Jul 25, 2014 at 4:32 PM, Bharath Ravi Kumar <reachb...@gmail.com> wrote: > Any suggestions to work around this issue ? The pre built spark binaries > don't appear to work against cdh as documented, unless there's a build > issue, which seems unlikely. > > On 25-Jul-2014 3:42 pm, "Bharath Ravi Kumar" <reachb...@gmail.com> wrote: >> >> >> I'm encountering a hadoop client protocol mismatch trying to read from >> HDFS (cdh3u5) using the pre-build spark from the downloads page (linked >> under "For Hadoop 1 (HDP1, CDH3)"). I've also followed the instructions at >> http://spark.apache.org/docs/latest/hadoop-third-party-distributions.html >> (i.e. building the app against hadoop-client 0.20.2-cdh3u5), but continue to >> see the following error regardless of whether I link the app with the cdh >> client: >> >> 14/07/25 09:53:43 INFO client.AppClient$ClientActor: Executor updated: >> app-20140725095343-0016/1 is now RUNNING >> 14/07/25 09:53:43 WARN util.NativeCodeLoader: Unable to load native-hadoop >> library for your platform... using builtin-java classes where applicable >> 14/07/25 09:53:43 WARN snappy.LoadSnappy: Snappy native library not loaded >> Exception in thread "main" org.apache.hadoop.ipc.RPC$VersionMismatch: >> Protocol org.apache.hadoop.hdfs.protocol.ClientProtocol version mismatch. >> (client = 61, server = 63) >> at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:401) >> at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:379) >> >> >> While I can build spark against the exact hadoop distro version, I'd >> rather work with the standard prebuilt binaries, making additional changes >> while building the app if necessary. Any workarounds/recommendations? >> >> Thanks, >> Bharath
