Spark - 1.3.1 Hbase - 1.0.0 Phoenix - 4.3 Cloudera - 5.4 On Tue, Dec 1, 2015 at 9:35 PM, Ted Yu <yuzhih...@gmail.com> wrote:
> What are the versions for Spark / HBase / Phoenix you're using ? > > Cheers > > On Tue, Dec 1, 2015 at 4:15 AM, Akhilesh Pathodia < > pathodia.akhil...@gmail.com> wrote: > >> Hi, >> >> I am running spark job on yarn in cluster mode in secured cluster. Spark >> executors are unable to get hbase connection using phoenix. I am running >> knit command to get the ticket before starting the job and also keytab file >> and principal are correctly specified in connection URL. But still spark >> job on each node throws below error: >> >> 15/12/01 03:23:15 ERROR ipc.AbstractRpcClient: SASL authentication >> failed. The most likely cause is missing or invalid credentials. Consider >> 'kinit'. >> javax.security.sasl.SaslException: GSS initiate failed [Caused by >> GSSException: No valid credentials provided (Mechanism level: Failed to >> find any Kerberos tgt)] >> at >> com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:212) >> at >> org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:179) >> at >> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupSaslConnection(RpcClientImpl.java:605) >> at >> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.access$600(RpcClientImpl.java:154) >> at >> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:731) >> at >> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:728) >> at java.security.AccessController.doPrivileged(Native Method) >> at javax.security.auth.Subject.doAs(Subject.java:415) >> at >> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671) >> at >> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupIOstreams(RpcClientImpl.java:728) >> at >> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.writeRequest(RpcClientImpl.java:881) >> at >> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.tracedWriteRequest(RpcClientImpl.java:850) >> at >> org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1174) >> at >> org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:216) >> at >> org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:300) >> at >> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$BlockingStub.isMasterRunning(MasterProtos.java:46470) >> at >> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$MasterServiceStubMaker.isMasterRunning(ConnectionManager.java:1606) >> at >> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$StubMaker.makeStubNoRetries(ConnectionManager.java:1544) >> at >> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$StubMaker.makeStub(ConnectionManager.java:1566) >> at >> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$MasterServiceStubMaker.makeStub(ConnectionManager.java:1595) >> at >> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getKeepAliveMasterService(ConnectionManager.java:1801) >> at >> org.apache.hadoop.hbase.client.MasterCallable.prepare(MasterCallable.java:38) >> at >> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:124) >> at >> org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:3678) >> at >> org.apache.hadoop.hbase.client.HBaseAdmin.getTableDescriptor(HBaseAdmin.java:451) >> at >> org.apache.hadoop.hbase.client.HBaseAdmin.getTableDescriptor(HBaseAdmin.java:473) >> at >> org.apache.phoenix.query.ConnectionQueryServicesImpl.ensureTableCreated(ConnectionQueryServicesImpl.java:804) >> at >> org.apache.phoenix.query.ConnectionQueryServicesImpl.createTable(ConnectionQueryServicesImpl.java:1194) >> at >> org.apache.phoenix.query.DelegateConnectionQueryServices.createTable(DelegateConnectionQueryServices.java:111) >> at >> org.apache.phoenix.schema.MetaDataClient.createTableInternal(MetaDataClient.java:1683) >> at >> org.apache.phoenix.schema.MetaDataClient.createTable(MetaDataClient.java:592) >> at >> org.apache.phoenix.compile.CreateTableCompiler$2.execute(CreateTableCompiler.java:177) >> at >> org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:280) >> at >> org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:272) >> at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53) >> at >> org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:270) >> at >> org.apache.phoenix.jdbc.PhoenixStatement.executeUpdate(PhoenixStatement.java:1052) >> at >> org.apache.phoenix.query.ConnectionQueryServicesImpl$11.call(ConnectionQueryServicesImpl.java:1841) >> at >> org.apache.phoenix.query.ConnectionQueryServicesImpl$11.call(ConnectionQueryServicesImpl.java:1810) >> at >> org.apache.phoenix.util.PhoenixContextExecutor.call(PhoenixContextExecutor.java:77) >> at >> org.apache.phoenix.query.ConnectionQueryServicesImpl.init(ConnectionQueryServicesImpl.java:1810) >> at >> org.apache.phoenix.jdbc.PhoenixDriver.getConnectionQueryServices(PhoenixDriver.java:162) >> at >> org.apache.phoenix.jdbc.PhoenixEmbeddedDriver.connect(PhoenixEmbeddedDriver.java:126) >> at >> org.apache.phoenix.jdbc.PhoenixDriver.connect(PhoenixDriver.java:133) >> at >> org.apache.commons.dbcp.DriverConnectionFactory.createConnection(DriverConnectionFactory.java:38) >> at >> org.apache.commons.dbcp.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:582) >> at >> org.apache.commons.dbcp.BasicDataSource.validateConnectionFactory(BasicDataSource.java:1556) >> >> >> >> I am able to connect to hbase using phoenix through a standalone >> application in secured server which means all the configurations required >> for getting the connection are correct. Do we need any addition >> configuration for making phoenix work in spark job running on yarn in >> cluster mode in secured cluster? >> >> How do make phoenix work with spark in secured environment? >> >> Thanks, >> Akhilesh Pathodia >> > >
