Suhas, Sqoop 1.4.3 simply fetches the authenticated user from credentials cache and fetches a delegation token for HBase. See https://issues.apache.org/jira/browse/SQOOP-599 for more information.
-Abe On Tue, Aug 6, 2013 at 11:09 AM, Suhas Satish <[email protected]>wrote: > I was able to isolate this problem to the Sqoop side not picking up > correct kerberos credentials. Hbase is picking up the correct kerberos > credentials when Hbase put and scan are done in isolation without using > Sqoop. > > A direct map-reduce put into HBase uses the following 2 methods - > HBaseConfiguration.merge(conf, HBaseConfiguration.create(conf)); > TableMapReduceUtil.initCredentials(job); > > I was looking at how sqoop 1.4.3 does HBase puts to see if it converts > sqoop import arguments into map-reduce jobs and uses the above methods > somewhere. This is what I found - > HBasePutProcessor.java - SqoopRecordProcessor that performs a HBase "put" > operation - has a method to get hadoop configuration, but none to merge any > kerberos specific configurations specified in sqoop-site.xml- > > public Configuration getConf() { > return this.conf; > > > > HBaseUtil.java - makes sure hbase jars are present on class path > PutTransformer.java - converts jdbc statements in the form of K-V map > into hbase put commands and returns a list > ToStringPutTransformer.java - extends the above class > > Does anyone know sqoop internals of how to specify kerberos > configurations and get sqoop to read them? > > Cheers, > Suhas. > > > On Tue, Aug 6, 2013 at 10:31 AM, Suhas Satish <[email protected]>wrote: > >> Ataching the logs here at the time of authentication, I do not see any >> error msges here. >> >> /var/log/kadmind.log >> /var/log/krb5kdc.log >> >> Please let me know if there is any other places I can find other log >> files >> >> Cheers, >> Suhas. >> >> >> On Mon, Aug 5, 2013 at 4:48 PM, Abraham Elmahrek <[email protected]>wrote: >> >>> User, >>> >>> Could you please provide your KDC logs around the time you tried to >>> authenticate? >>> >>> Note: A kerberos client will negotiate the encryption algorithm it >>> can/will use with the KDC. It may choose AES-256. >>> >>> -Abe >>> >>> >>> On Mon, Aug 5, 2013 at 3:55 PM, Suhas Satish <[email protected]>wrote: >>> >>>> I generated a keytab with the following cmd and it supports multiple >>>> encryption types other than aes256 as listed below. >>>> But I still get the same error from sqoop import tool because the >>>> sqoop.keytab is not being read (sqoop being the hbase client in this case). >>>> >>>> kadmin: ktadd -k sqoop.keytab kuser1 >>>> Entry for principal kuser1 with kvno 2, encryption type >>>> aes256-cts-hmac-sha1-96 added to keytab WRFILE:sqoop.keytab. >>>> Entry for principal kuser1 with kvno 2, encryption type >>>> aes128-cts-hmac-sha1-96 added to keytab WRFILE:sqoop.keytab. >>>> Entry for principal kuser1 with kvno 2, encryption type des3-cbc-sha1 >>>> added to keytab WRFILE:sqoop.keytab. >>>> Entry for principal kuser1 with kvno 2, encryption type arcfour-hmac >>>> added to keytab WRFILE:sqoop.keytab. >>>> Entry for principal kuser1 with kvno 2, encryption type des-hmac-sha1 >>>> added to keytab WRFILE:sqoop.keytab. >>>> Entry for principal kuser1 with kvno 2, encryption type des-cbc-md5 >>>> added to keytab WRFILE:sqoop.keytab. >>>> >>>> Here are some more debug logs I obtained from kerberos - >>>> >>>> *kadmin: getprinc kuser1* >>>> Principal: [email protected] >>>> Expiration date: [never] >>>> Last password change: Mon Aug 05 15:40:30 PDT 2013 >>>> Password expiration date: [none] >>>> Maximum ticket life: 1 day 00:00:00 >>>> Maximum renewable life: 0 days 00:00:00 >>>> Last modified: Mon Aug 05 15:40:30 PDT 2013 (mapr/[email protected]) >>>> Last successful authentication: [never] >>>> Last failed authentication: [never] >>>> Failed password attempts: 0 >>>> Number of keys: 6 >>>> Key: vno 2, aes256-cts-hmac-sha1-96, no salt >>>> Key: vno 2, aes128-cts-hmac-sha1-96, no salt >>>> Key: vno 2, des3-cbc-sha1, no salt >>>> Key: vno 2, arcfour-hmac, no salt >>>> Key: vno 2, des-hmac-sha1, no salt >>>> Key: vno 2, des-cbc-md5, no salt >>>> MKey: vno 1 >>>> Attributes: >>>> Policy: [none] >>>> >>>> *getprinc hbase/qa-node133.qa.lab* >>>> Principal: hbase/[email protected] >>>> Expiration date: [never] >>>> Last password change: Mon Jul 29 19:17:46 PDT 2013 >>>> Password expiration date: [none] >>>> Maximum ticket life: 0 days 10:00:00 >>>> Maximum renewable life: 7 days 00:00:00 >>>> Last modified: Mon Jul 29 19:17:46 PDT 2013 (kuser1/[email protected]) >>>> Last successful authentication: [never] >>>> Last failed authentication: [never] >>>> Failed password attempts: 0 >>>> Number of keys: 6 >>>> Key: vno 2, aes256-cts-hmac-sha1-96, no salt >>>> Key: vno 2, aes128-cts-hmac-sha1-96, no salt >>>> Key: vno 2, des3-cbc-sha1, no salt >>>> Key: vno 2, arcfour-hmac, no salt >>>> Key: vno 2, des-hmac-sha1, no salt >>>> Key: vno 2, des-cbc-md5, no salt >>>> MKey: vno 1 >>>> Attributes: >>>> Policy: [none] >>>> >>>> >>>> Thanks, >>>> Suhas. >>>> >>>> >>>> On Mon, Aug 5, 2013 at 2:29 PM, Abraham Elmahrek <[email protected]>wrote: >>>> >>>>> There should be a password. You should have a keytab associated with >>>>> that principal, which would allow you to authenticate as that principal. >>>>> See >>>>> http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH4/latest/CDH4-Security-Guide/CDH4-Security-Guide.htmlfor >>>>> more details on how that works. >>>>> >>>>> A couple of things... >>>>> 1. You need to make your kerberos credentials renewable. Right now it >>>>> seems like you cannot renew. See >>>>> http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH4/latest/CDH4-Security-Guide/cdh4sg_topic_17.html >>>>> . >>>>> 2. AES256 encryption is not inherently supported. Did you install >>>>> support for AES256? >>>>> >>>>> -Abe >>>>> >>>>> >>>>> On Mon, Aug 5, 2013 at 1:53 PM, Suhas Satish >>>>> <[email protected]>wrote: >>>>> >>>>>> klist -e -v >>>>>> >>>>>> Ticket cache: FILE:/tmp/krb5cc_0 >>>>>> Default principal: [email protected] >>>>>> >>>>>> Valid starting Expires Service principal >>>>>> 08/05/13 12:34:42 08/05/13 22:34:42 krbtgt/[email protected] >>>>>> renew until 08/05/13 12:34:42, Etype (skey, tkt): >>>>>> aes256-cts-hmac-sha1-96, aes256-cts-hmac-sha1-96 >>>>>> >>>>>> Kerberos 5 version 1.10.3 >>>>>> >>>>>> The principal in hbase-site.xml is >>>>>> hbase/[email protected] >>>>>> >>>>>> How do I create a credential using kinit matching that in >>>>>> hbase-site.xml? kinit hbase/qa-node133.qa.lab throws an error msg >>>>>> *kinit: Password incorrect while getting initial credentials* >>>>>> *although I know that there is no password for that principal. * >>>>>> * >>>>>> * >>>>>> * >>>>>> * >>>>>> >>>>>> Cheers, >>>>>> Suhas. >>>>>> >>>>>> >>>>>> On Mon, Aug 5, 2013 at 12:52 PM, Abraham Elmahrek >>>>>> <[email protected]>wrote: >>>>>> >>>>>>> Hi there, >>>>>>> >>>>>>> It seems like your client isn't authenticated in both cases. You >>>>>>> seem to be receiving errors from HBase and Sqoop. Sqoop 1.4.3 should >>>>>>> simply >>>>>>> work if your user is already authenticated. Internally, Sqoop is >>>>>>> generating >>>>>>> delegation tokens to communicate with HBase. It cannot do that without >>>>>>> being properly authenticated first though. >>>>>>> >>>>>>> Could you provide the output of the following command: >>>>>>> "klist -e -v" >>>>>>> >>>>>>> -Abe >>>>>>> >>>>>>> >>>>>>> On Mon, Aug 5, 2013 at 12:15 PM, Suhas Satish < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> I have configured hbase 94.9 with kerberos successfully for >>>>>>>> authentication and authorization as mentioned in the CDH security >>>>>>>> docs. I >>>>>>>> am using sqoop 1.4.3. Is there any configuration required from the >>>>>>>> sqoop >>>>>>>> client side for kerberos? >>>>>>>> >>>>>>>> I have the following permissions on hbase tables - >>>>>>>> hbase(main):003:0> grant 'kuser1', 'RWXCA', 'demo' >>>>>>>> ERROR: org.apache.hadoop.hbase.security.AccessDeniedException: >>>>>>>> org.apache.hadoop.hbase.security.AccessDeniedException: Insufficient >>>>>>>> permissions (user=kuser1, scope=demo, family=, qualifer=, action=ADMIN) >>>>>>>> >>>>>>>> >>>>>>>> bin/sqoop import --connect jdbc:mysql://10.10.1.10/TestDB >>>>>>>> --table t1 --hbase-table t1 --column-family world >>>>>>>> >>>>>>>> >>>>>>>> When I try to import into it using sqoop with the above cmd, I get >>>>>>>> the following error - >>>>>>>> >>>>>>>> >>>>>>>> 2013-08-05 11:59:33,121 ERROR >>>>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer: >>>>>>>> org.apache.hadoop.hbase.security.AccessDeniedException: Token >>>>>>>> generation only allowed for Kerberos authenticated clients >>>>>>>> at >>>>>>>> org.apache.hadoop.hbase.security.token.TokenProvider.getAuthenticationToken(TokenProvider.java:87) >>>>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>>>>>>> at >>>>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) >>>>>>>> at >>>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) >>>>>>>> at java.lang.reflect.Method.invoke(Method.java:597) >>>>>>>> at >>>>>>>> org.apache.hadoop.hbase.regionserver.HRegion.exec(HRegion.java:5576) >>>>>>>> at >>>>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer.execCoprocessor(HRegionServer.java:3868) >>>>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>>>>>>> at >>>>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) >>>>>>>> at >>>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) >>>>>>>> at java.lang.reflect.Method.invoke(Method.java:597) >>>>>>>> at >>>>>>>> org.apache.hadoop.hbase.ipc.SecureRpcEngine$Server.call(SecureRpcEngine.java:308) >>>>>>>> at >>>>>>>> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1426) >>>>>>>> >>>>>>>> >>>>>>>> Cheers, >>>>>>>> Suhas. >>>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> >
