Hi

I also have faced the same problem with shark 0.9.1 version and i have it
fixed by sbt clean/packaging the shark with the rite hadoop version. You
may execute the following commands to get it done.

*cd shark;export SHARK_HADOOP_VERSION=$(/root/ephemeral-hdfs/bin/hadoop
version | head -n1 | cut -d" " -f 2-);sbt/sbt clean*

*cd shark;export SHARK_HADOOP_VERSION=$(/root/ephemeral-hdfs/bin/hadoop
version | head -n1 | cut -d" " -f 2-);sbt/sbt package*




On Fri, Apr 25, 2014 at 11:16 PM, jesseerdmann <jerdm...@umn.edu> wrote:

> I've run into a problem trying to launch a cluster using the provided ec2
> python script with --hadoop-major-version 2.  The launch completes
> correctly
> with the exception of an Exception getting thrown for Tachyon 7 (I've
> included it at the end of the message, but that is not the focus and seems
> unrelated to my issue.)
>
> When I log in and try to run shark-withinfo, I get the following exception
> and I'm not sure where to go from here.
>
> Exception in thread "main" java.lang.RuntimeException:
> org.apache.hadoop.hive.ql.metadata.HiveException:
> java.lang.RuntimeException: java.lang.RuntimeException:
> java.lang.reflect.InvocationTargetException
>         at
> org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:278)
>         at shark.SharkCliDriver$.main(SharkCliDriver.scala:128)
>         at shark.SharkCliDriver.main(SharkCliDriver.scala)
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
> java.lang.RuntimeException: java.lang.RuntimeException:
> java.lang.reflect.InvocationTargetException
>         at
>
> org.apache.hadoop.hive.ql.metadata.HiveUtils.getAuthenticator(HiveUtils.java:368)
>         at
> org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:270)
>         ... 2 more
> Caused by: java.lang.RuntimeException: java.lang.RuntimeException:
> java.lang.reflect.InvocationTargetException
>         at
>
> org.apache.hadoop.hive.ql.security.HadoopDefaultAuthenticator.setConf(HadoopDefaultAuthenticator.java:53)
>         at
> org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:73)
>         at
>
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133)
>         at
>
> org.apache.hadoop.hive.ql.metadata.HiveUtils.getAuthenticator(HiveUtils.java:365)
>         ... 3 more
> Caused by: java.lang.RuntimeException:
> java.lang.reflect.InvocationTargetException
>         at
>
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:131)
>         at org.apache.hadoop.security.Groups.<init>(Groups.java:64)
>         at
>
> org.apache.hadoop.security.Groups.getUserToGroupsMappingService(Groups.java:240)
>         at
>
> org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:255)
>         at
>
> org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:232)
>         at
>
> org.apache.hadoop.security.UserGroupInformation.loginUserFromSubject(UserGroupInformation.java:718)
>         at
>
> org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:703)
>         at
>
> org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:605)
>         at
>
> org.apache.hadoop.hive.shims.HadoopShimsSecure.getUGIForConf(HadoopShimsSecure.java:491)
>         at
>
> org.apache.hadoop.hive.ql.security.HadoopDefaultAuthenticator.setConf(HadoopDefaultAuthenticator.java:51)
>         ... 6 more
> Caused by: java.lang.reflect.InvocationTargetException
>         at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
>         at
>
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
>         at
>
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
>         at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
>         at
>
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:129)
>         ... 15 more
> Caused by: java.lang.UnsatisfiedLinkError:
> org.apache.hadoop.security.JniBasedUnixGroupsMapping.anchorNative()V
>         at
> org.apache.hadoop.security.JniBasedUnixGroupsMapping.anchorNative(Native
> Method)
>         at
>
> org.apache.hadoop.security.JniBasedUnixGroupsMapping.<clinit>(JniBasedUnixGroupsMapping.java:49)
>         at
>
> org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback.<init>(JniBasedUnixGroupsMappingWithFallback.java:38)
>         ... 20 more
>
>
>
>
>
> For completeness, the Tachyon exception during cluster launch:
>
> Exception in thread "main" java.lang.RuntimeException:
> org.apache.hadoop.ipc.RemoteException: Server IPC version 7 cannot
> communicate with client version 4
>         at tachyon.util.CommonUtils.runtimeException(CommonUtils.java:246)
>         at tachyon.UnderFileSystemHdfs.<init>(UnderFileSystemHdfs.java:73)
>         at
> tachyon.UnderFileSystemHdfs.getClient(UnderFileSystemHdfs.java:53)
>         at tachyon.UnderFileSystem.get(UnderFileSystem.java:53)
>         at tachyon.Format.main(Format.java:54)
> Caused by: org.apache.hadoop.ipc.RemoteException: Server IPC version 7
> cannot communicate with client version 4
>         at org.apache.hadoop.ipc.Client.call(Client.java:1070)
>         at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:225)
>         at com.sun.proxy.$Proxy1.getProtocolVersion(Unknown Source)
>         at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:396)
>         at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:379)
>         at
> org.apache.hadoop.hdfs.DFSClient.createRPCNamenode(DFSClient.java:119)
>         at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:238)
>         at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:203)
>         at
>
> org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:89)
>         at
> org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1386)
>         at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:66)
>         at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1404)
>         at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:254)
>         at org.apache.hadoop.fs.Path.getFileSystem(Path.java:187)
>         at tachyon.UnderFileSystemHdfs.<init>(UnderFileSystemHdfs.java:69)
>         ... 3 more
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Shark-0-9-1-on-ec2-with-Hadoop-2-error-tp4837.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>

Reply via email to