x-posting to Hadoop See following error. Hadoop version is 2.3 (CDH 5.0). Name node and Resource Manager are in HA configuration
Any thoughts ? Thanks, ---------- Forwarded message ---------- From: Manoj Samel <[email protected]> Date: Thu, Jan 8, 2015 at 7:33 PM Subject: Re: Running spark 1.2 on Hadoop + Kerberos To: Marcelo Vanzin <[email protected]> After doing kinit, when the job is submitted using spark-submit, it gives following trace. Any idea what is the issue ? HDFS is up and running 15/01/09 03:21:54 INFO yarn.Client: Will allocate AM container, with 1408 MB memory including 384 MB overhead 15/01/09 03:21:54 INFO yarn.Client: Setting up container launch context for our AM 15/01/09 03:21:54 INFO yarn.Client: Preparing resources for our AM container 15/01/09 03:21:55 INFO hdfs.DFSClient: Created HDFS_DELEGATION_TOKEN token 112908 for xxx on xxx:8020 Exception in thread "main" org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.ipc.StandbyException): Operation category WRITE is not supported in state standby at org.apache.hadoop.hdfs.server.namenode.ha.StandbyState.checkOperation(StandbyState.java:87) at org.apache.hadoop.hdfs.server.namenode.NameNode$NameNodeHAContext.checkOperation(NameNode.java:1565) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkOperation(FSNamesystem.java:1181) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getDelegationToken(FSNamesystem.java:6205) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getDelegationToken(NameNodeRpcServer.java:461) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getDelegationToken(ClientNamenodeProtocolServerSideTranslatorPB.java:905) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) On Thu, Jan 8, 2015 at 4:15 PM, Marcelo Vanzin <[email protected]> wrote: > On Thu, Jan 8, 2015 at 4:09 PM, Manoj Samel <[email protected]> > wrote: > > Some old communication (Oct 14) says Spark is not certified with > Kerberos. > > Can someone comment on this aspect ? > > Spark standalone doesn't support kerberos. Spark running on top of > Yarn works fine with kerberos. > > -- > Marcelo >
