Sorry, apparently this is an HBase specific token. See here http://wiki.apache.org/hadoop/Hbase/HBaseTokenAuthentication.
On Tue, Aug 6, 2013 at 11:13 AM, Abraham Elmahrek <[email protected]> wrote: > Suhas, > > Sqoop 1.4.3 simply fetches the authenticated user from credentials cache > and fetches a delegation token for HBase. See > https://issues.apache.org/jira/browse/SQOOP-599 for more information. > > -Abe > > > On Tue, Aug 6, 2013 at 11:09 AM, Suhas Satish <[email protected]>wrote: > >> I was able to isolate this problem to the Sqoop side not picking up >> correct kerberos credentials. Hbase is picking up the correct kerberos >> credentials when Hbase put and scan are done in isolation without using >> Sqoop. >> >> A direct map-reduce put into HBase uses the following 2 methods - >> HBaseConfiguration.merge(conf, HBaseConfiguration.create(conf)); >> TableMapReduceUtil.initCredentials(job); >> >> I was looking at how sqoop 1.4.3 does HBase puts to see if it converts >> sqoop import arguments into map-reduce jobs and uses the above methods >> somewhere. This is what I found - >> HBasePutProcessor.java - SqoopRecordProcessor that performs a HBase >> "put" operation - has a method to get hadoop configuration, but none to >> merge any kerberos specific configurations specified in sqoop-site.xml- >> >> public Configuration getConf() { >> return this.conf; >> >> >> >> HBaseUtil.java - makes sure hbase jars are present on class path >> PutTransformer.java - converts jdbc statements in the form of K-V map >> into hbase put commands and returns a list >> ToStringPutTransformer.java - extends the above class >> >> Does anyone know sqoop internals of how to specify kerberos >> configurations and get sqoop to read them? >> >> Cheers, >> Suhas. >> >> >> On Tue, Aug 6, 2013 at 10:31 AM, Suhas Satish <[email protected]>wrote: >> >>> Ataching the logs here at the time of authentication, I do not see any >>> error msges here. >>> >>> /var/log/kadmind.log >>> /var/log/krb5kdc.log >>> >>> Please let me know if there is any other places I can find other log >>> files >>> >>> Cheers, >>> Suhas. >>> >>> >>> On Mon, Aug 5, 2013 at 4:48 PM, Abraham Elmahrek <[email protected]>wrote: >>> >>>> User, >>>> >>>> Could you please provide your KDC logs around the time you tried to >>>> authenticate? >>>> >>>> Note: A kerberos client will negotiate the encryption algorithm it >>>> can/will use with the KDC. It may choose AES-256. >>>> >>>> -Abe >>>> >>>> >>>> On Mon, Aug 5, 2013 at 3:55 PM, Suhas Satish <[email protected]>wrote: >>>> >>>>> I generated a keytab with the following cmd and it supports multiple >>>>> encryption types other than aes256 as listed below. >>>>> But I still get the same error from sqoop import tool because the >>>>> sqoop.keytab is not being read (sqoop being the hbase client in this >>>>> case). >>>>> >>>>> kadmin: ktadd -k sqoop.keytab kuser1 >>>>> Entry for principal kuser1 with kvno 2, encryption type >>>>> aes256-cts-hmac-sha1-96 added to keytab WRFILE:sqoop.keytab. >>>>> Entry for principal kuser1 with kvno 2, encryption type >>>>> aes128-cts-hmac-sha1-96 added to keytab WRFILE:sqoop.keytab. >>>>> Entry for principal kuser1 with kvno 2, encryption type des3-cbc-sha1 >>>>> added to keytab WRFILE:sqoop.keytab. >>>>> Entry for principal kuser1 with kvno 2, encryption type arcfour-hmac >>>>> added to keytab WRFILE:sqoop.keytab. >>>>> Entry for principal kuser1 with kvno 2, encryption type des-hmac-sha1 >>>>> added to keytab WRFILE:sqoop.keytab. >>>>> Entry for principal kuser1 with kvno 2, encryption type des-cbc-md5 >>>>> added to keytab WRFILE:sqoop.keytab. >>>>> >>>>> Here are some more debug logs I obtained from kerberos - >>>>> >>>>> *kadmin: getprinc kuser1* >>>>> Principal: [email protected] >>>>> Expiration date: [never] >>>>> Last password change: Mon Aug 05 15:40:30 PDT 2013 >>>>> Password expiration date: [none] >>>>> Maximum ticket life: 1 day 00:00:00 >>>>> Maximum renewable life: 0 days 00:00:00 >>>>> Last modified: Mon Aug 05 15:40:30 PDT 2013 (mapr/[email protected]) >>>>> Last successful authentication: [never] >>>>> Last failed authentication: [never] >>>>> Failed password attempts: 0 >>>>> Number of keys: 6 >>>>> Key: vno 2, aes256-cts-hmac-sha1-96, no salt >>>>> Key: vno 2, aes128-cts-hmac-sha1-96, no salt >>>>> Key: vno 2, des3-cbc-sha1, no salt >>>>> Key: vno 2, arcfour-hmac, no salt >>>>> Key: vno 2, des-hmac-sha1, no salt >>>>> Key: vno 2, des-cbc-md5, no salt >>>>> MKey: vno 1 >>>>> Attributes: >>>>> Policy: [none] >>>>> >>>>> *getprinc hbase/qa-node133.qa.lab* >>>>> Principal: hbase/[email protected] >>>>> Expiration date: [never] >>>>> Last password change: Mon Jul 29 19:17:46 PDT 2013 >>>>> Password expiration date: [none] >>>>> Maximum ticket life: 0 days 10:00:00 >>>>> Maximum renewable life: 7 days 00:00:00 >>>>> Last modified: Mon Jul 29 19:17:46 PDT 2013 (kuser1/[email protected]) >>>>> Last successful authentication: [never] >>>>> Last failed authentication: [never] >>>>> Failed password attempts: 0 >>>>> Number of keys: 6 >>>>> Key: vno 2, aes256-cts-hmac-sha1-96, no salt >>>>> Key: vno 2, aes128-cts-hmac-sha1-96, no salt >>>>> Key: vno 2, des3-cbc-sha1, no salt >>>>> Key: vno 2, arcfour-hmac, no salt >>>>> Key: vno 2, des-hmac-sha1, no salt >>>>> Key: vno 2, des-cbc-md5, no salt >>>>> MKey: vno 1 >>>>> Attributes: >>>>> Policy: [none] >>>>> >>>>> >>>>> Thanks, >>>>> Suhas. >>>>> >>>>> >>>>> On Mon, Aug 5, 2013 at 2:29 PM, Abraham Elmahrek <[email protected]>wrote: >>>>> >>>>>> There should be a password. You should have a keytab associated with >>>>>> that principal, which would allow you to authenticate as that principal. >>>>>> See >>>>>> http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH4/latest/CDH4-Security-Guide/CDH4-Security-Guide.htmlfor >>>>>> more details on how that works. >>>>>> >>>>>> A couple of things... >>>>>> 1. You need to make your kerberos credentials renewable. Right now it >>>>>> seems like you cannot renew. See >>>>>> http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH4/latest/CDH4-Security-Guide/cdh4sg_topic_17.html >>>>>> . >>>>>> 2. AES256 encryption is not inherently supported. Did you install >>>>>> support for AES256? >>>>>> >>>>>> -Abe >>>>>> >>>>>> >>>>>> On Mon, Aug 5, 2013 at 1:53 PM, Suhas Satish >>>>>> <[email protected]>wrote: >>>>>> >>>>>>> klist -e -v >>>>>>> >>>>>>> Ticket cache: FILE:/tmp/krb5cc_0 >>>>>>> Default principal: [email protected] >>>>>>> >>>>>>> Valid starting Expires Service principal >>>>>>> 08/05/13 12:34:42 08/05/13 22:34:42 krbtgt/[email protected] >>>>>>> renew until 08/05/13 12:34:42, Etype (skey, tkt): >>>>>>> aes256-cts-hmac-sha1-96, aes256-cts-hmac-sha1-96 >>>>>>> >>>>>>> Kerberos 5 version 1.10.3 >>>>>>> >>>>>>> The principal in hbase-site.xml is >>>>>>> hbase/[email protected] >>>>>>> >>>>>>> How do I create a credential using kinit matching that in >>>>>>> hbase-site.xml? kinit hbase/qa-node133.qa.lab throws an error msg >>>>>>> *kinit: Password incorrect while getting initial credentials* >>>>>>> *although I know that there is no password for that principal. * >>>>>>> * >>>>>>> * >>>>>>> * >>>>>>> * >>>>>>> >>>>>>> Cheers, >>>>>>> Suhas. >>>>>>> >>>>>>> >>>>>>> On Mon, Aug 5, 2013 at 12:52 PM, Abraham Elmahrek >>>>>>> <[email protected]>wrote: >>>>>>> >>>>>>>> Hi there, >>>>>>>> >>>>>>>> It seems like your client isn't authenticated in both cases. You >>>>>>>> seem to be receiving errors from HBase and Sqoop. Sqoop 1.4.3 should >>>>>>>> simply >>>>>>>> work if your user is already authenticated. Internally, Sqoop is >>>>>>>> generating >>>>>>>> delegation tokens to communicate with HBase. It cannot do that without >>>>>>>> being properly authenticated first though. >>>>>>>> >>>>>>>> Could you provide the output of the following command: >>>>>>>> "klist -e -v" >>>>>>>> >>>>>>>> -Abe >>>>>>>> >>>>>>>> >>>>>>>> On Mon, Aug 5, 2013 at 12:15 PM, Suhas Satish < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> I have configured hbase 94.9 with kerberos successfully for >>>>>>>>> authentication and authorization as mentioned in the CDH security >>>>>>>>> docs. I >>>>>>>>> am using sqoop 1.4.3. Is there any configuration required from the >>>>>>>>> sqoop >>>>>>>>> client side for kerberos? >>>>>>>>> >>>>>>>>> I have the following permissions on hbase tables - >>>>>>>>> hbase(main):003:0> grant 'kuser1', 'RWXCA', 'demo' >>>>>>>>> ERROR: org.apache.hadoop.hbase.security.AccessDeniedException: >>>>>>>>> org.apache.hadoop.hbase.security.AccessDeniedException: Insufficient >>>>>>>>> permissions (user=kuser1, scope=demo, family=, qualifer=, >>>>>>>>> action=ADMIN) >>>>>>>>> >>>>>>>>> >>>>>>>>> bin/sqoop import --connect jdbc:mysql://10.10.1.10/TestDB >>>>>>>>> --table t1 --hbase-table t1 --column-family world >>>>>>>>> >>>>>>>>> >>>>>>>>> When I try to import into it using sqoop with the above cmd, I get >>>>>>>>> the following error - >>>>>>>>> >>>>>>>>> >>>>>>>>> 2013-08-05 11:59:33,121 ERROR >>>>>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer: >>>>>>>>> org.apache.hadoop.hbase.security.AccessDeniedException: Token >>>>>>>>> generation only allowed for Kerberos authenticated clients >>>>>>>>> at >>>>>>>>> org.apache.hadoop.hbase.security.token.TokenProvider.getAuthenticationToken(TokenProvider.java:87) >>>>>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>>>>>>>> at >>>>>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) >>>>>>>>> at >>>>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) >>>>>>>>> at java.lang.reflect.Method.invoke(Method.java:597) >>>>>>>>> at >>>>>>>>> org.apache.hadoop.hbase.regionserver.HRegion.exec(HRegion.java:5576) >>>>>>>>> at >>>>>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer.execCoprocessor(HRegionServer.java:3868) >>>>>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>>>>>>>> at >>>>>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) >>>>>>>>> at >>>>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) >>>>>>>>> at java.lang.reflect.Method.invoke(Method.java:597) >>>>>>>>> at >>>>>>>>> org.apache.hadoop.hbase.ipc.SecureRpcEngine$Server.call(SecureRpcEngine.java:308) >>>>>>>>> at >>>>>>>>> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1426) >>>>>>>>> >>>>>>>>> >>>>>>>>> Cheers, >>>>>>>>> Suhas. >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> >
