Hello, Sqoop 1.4.6 support kerberos ==> it mean your mapreduce process is lanched by a *principal *that is in realm of KDC (entry point) , verify ticket with klist so you dont need to spécify options integratedSecurity=true; authenticationScheme=JavaKerberos;
hope this helps you sqoop import --connect "jdbc:sqlserver://FQDN;database=database_name" --username 'Domain\user' --query 'select * from table where $CONDITIONS' --split-by Abc_Number --target-dir /user/user/SQL 2016-09-07 17:47 GMT+02:00 Jonathan Kaufman <[email protected]>: > Hello all, > > I apologize in advance if this seems like a silly question, however I've > not been able to determine if Scoop supports Kerberos delegation on > import, down to the MapReduce process. > > I have a user who has a valid Kerberos ticket that supports forwarding, > additionally they are working on a completely kerberized Hadoop cluster and > all hosts in the cluster are trusted for kerberos delegation. > > We have been able to use Scoop with Kerberos Authentication when > connecting to a Microsoft SQL Server using the following command: > > sqoop > list-databases --connect > "jdbc:sqlserver://hostname.domain.com;integratedSecurity= > true;authenticationScheme=JavaKerberos" > --username 'DOMAIN\USER' > > So we tried to do an import using Kerberos, but it has failed with an > error that indicates that MapReduce does not have a kerberos ticket. > > Here is the command used on the Sqoop import. > > sqoop import --connect "jdbc:sqlserver://FQDN;integratedSecurity=true; > authenticationScheme=JavaKerberos;database=database_name" --username > 'Domain\user' --query 'select * from table where $CONDITIONS' --split-by > Abc_Number --target-dir /user/user/Sql > > Here is the error: > INFO mapreduce.JobSubmitter: Kind: HDFS_DELEGATION_TOKEN, Service: > ha-hdfs:NNHA, Ident: (HDFS_DELEGATION_TOKEN token 2496 for ) INFO > impl.YarnClientImpl: Submitted application INFO mapreduce.Job: The url to > track the job: INFO mapreduce.Job: Running job: job_1 INFO mapreduce.Job: > Job job_1 running in uber mode : false INFO mapreduce.Job: map 0% reduce 0% > INFO mapreduce.Job: Task Id : attempt_1470416754637_0238_m_000003_0, > Status : FAILED Error: java.lang.RuntimeException: > java.lang.RuntimeException: com.microsoft.sqlserver.jdbc.SQLServerException: > Integrated authentication failed. ClientConnectionId:366c38c7-c at > org.apache.sqoop.mapreduce.db.DBInputFormat.setConf(DBInputFormat.java:167) > at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76) > at > org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136) > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:749) at > org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) at > org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168) at > java.security.AccessController.doPrivileged(Native Method) at > javax.security.auth.Subject.doAs(Subject.java:422) at > org.apache.hadoop.security.UserGroupInformation.doAs( > UserGroupInformation.java:1709) at org.apache.hadoop.mapred. > YarnChild.main(YarnChild.java:162) Caused by: java.lang.RuntimeException: > com.microsoft.sqlserver.jdbc.SQLServerException: Integrated > authentication failed. ClientConnectionId:366c38c7-c at > org.apache.sqoop.mapreduce.db.DBInputFormat.getConnection(DBInputFormat.java:220) > at org.apache.sqoop.mapreduce.db.DBInputFormat.setConf(DBInputFormat.java:165) > ... 9 more Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: > Integrated authentication failed. ClientConnectionId:366c38c7-c at > com.microsoft.sqlserver.jdbc.SQLServerConnection.terminate(SQLServerConnection.java:2399) > at > com.microsoft.sqlserver.jdbc.KerbAuthentication.intAuthInit(KerbAuthentication.java:176) > at > com.microsoft.sqlserver.jdbc.KerbAuthentication.GenerateClientContext(KerbAuthentication.java:296) > at > com.microsoft.sqlserver.jdbc.SQLServerConnection.sendLogon(SQLServerConnection.java:4084) > at > com.microsoft.sqlserver.jdbc.SQLServerConnection.logon(SQLServerConnection.java:3159) > at com.microsoft.sqlserver.jdbc.SQLServerConnection.access$ > 100(SQLServerConnection.java:42) at com.microsoft.sqlserver.jdbc. > SQLServerConnection$LogonCommand.doExecute(SQLServerConnection.java:3122) > at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:7505) at > com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:2444) > at > com.microsoft.sqlserver.jdbc.SQLServerConnection.connectHelper(SQLServerConnection.java:1980) > at > com.microsoft.sqlserver.jdbc.SQLServerConnection.login(SQLServerConnection.java:1627) > at > com.microsoft.sqlserver.jdbc.SQLServerConnection.connectInternal(SQLServerConnection.java:1458) > at > com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(SQLServerConnection.java:772) > at > com.microsoft.sqlserver.jdbc.SQLServerDriver.connect(SQLServerDriver.java:1168) > at java.sql.DriverManager.getConnection(DriverManager.java:664) at > java.sql.DriverManager.getConnection(DriverManager.java:247) at > org.apache.sqoop.mapreduce.db.DBConfiguration.getConnection(DBConfiguration.java:302) > at > org.apache.sqoop.mapreduce.db.DBInputFormat.getConnection(DBInputFormat.java:213) > ... 10 more Caused by: java.security.PrivilegedActionException: > GSSException: No valid credentials provided (Mechanism level: Failed to > find any Kerberos tgt) at java.security.AccessController.doPrivileged(Native > Method) at javax.security.auth.Subject.doAs(Subject.java:422) at > com.microsoft.sqlserver.jdbc.KerbAuthentication.getClientCredential(KerbAuthentication.java:199) > at > com.microsoft.sqlserver.jdbc.KerbAuthentication.intAuthInit(KerbAuthentication.java:150) > ... 26 more Caused by: GSSException: No valid credentials provided > (Mechanism level: Failed to find any Kerberos tgt) at > sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:147) > at > sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5MechFactory.java:122) > at > sun.security.jgss.GSSManagerImpl.getCredentialElement(GSSManagerImpl.java:193) > at sun.security.jgss.GSSCredentialImpl.add(GSSCredentialImpl.java:427) at > sun.security.jgss.GSSCredentialImpl.<init>(GSSCredentialImpl.java:62) at > sun.security.jgss.GSSManagerImpl.createCredential(GSSManagerImpl.java:154) > at > com.microsoft.sqlserver.jdbc.KerbAuthentication$1.run(KerbAuthentication.java:189) > at > com.microsoft.sqlserver.jdbc.KerbAuthentication$1.run(KerbAuthentication.java:187) > ... 30 more > > > Does anyone know if Scoop support forwarding the Kerberos ticket? and if > so, what do we need to configure to have it do so? Any comments on how this > should work, or websites discussing how you got this working in your > environment? > > There is a Hortonworks Community post on this as well: https://community. > hortonworks.com/questions/52137/sqlserver-sqoop-import- > integrated-security-kerbero.html > > I would like to verify each step of the process can support Kerberos, and > then verify the configurations to determine if there is any configuration > set that might prevent this from working. > > Any help would be appreciated. > > thanks. > > Jonathan Kaufman | DevOps Engineer | Foot Locker Inc. > (414-357-4062) | [email protected] > > Visit us on-line at *footlocker.com* > <http://www.footlocker.com/home/default.cfm?SID=8431>. > > > The information in this e-mail, and any attachment therein, is > confidential and for use by the addressee only. If you are not the intended > recipient, please return the e-mail to the sender and delete it from your > computer. Although the Company attempts to sweep e-mail and attachments for > viruses, it does not guarantee that either are virus-free and accepts no > liability for any damage sustained as a result of viruses. -- Rejoignez moi sur le réseau professionnel viadeo à l'adresse: http://www.viadeo.com/fr/profile/matouk.iftissen1
