Dilip Thallam Sridhar created SPARK-36765:
---------------------------------------------
Summary: Support MS Sql JDBC connectorJDBC with Keytab
Key: SPARK-36765
URL: https://issues.apache.org/jira/browse/SPARK-36765
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 3.1.2
Environment: Unix Redhat Environment
Reporter: Dilip Thallam Sridhar
Fix For: 3.1.2
Hi Team,
We are using the Spark-3.0.2 to connect to MS SqlServer with the following
instruction
Also tried with the Spark-3.1.2 Version,
1) download mssql-jdbc-9.4.0.jre8.jar
2) Generated Keytab using kinit
3) Validate Keytab using klist
4) Run the spark job with principal and keytabs passed
5) connection_url =
"jdbc:sqlserver://{}:{};databaseName={};integratedSecurity=true;authenticationSchema=JavaKerberos"\
.format(jdbc_host_name, jdbc_port, jdbc_database_name)
Note: without integratedSecurity=true;authenticationSchema=JavaKerberos it
looks for the usual username/password option to connect
6) passing the following options during spark read.
.option("principal", database_principal) \
.option("files", database_keytab) \
.option("keytab", database_keytab) \
tried with files and keytab, just files, and with all above 3 parameters
We are unable to connect to SqlServer from Spark and getting the following
error shown below.
A) Wanted to know if anybody was successful Spark to SqlServer? (as I see the
previous Jira has been closed)
https://issues.apache.org/jira/browse/SPARK-12312
https://issues.apache.org/jira/browse/SPARK-31337
B) If yes, could you let us know if there are any additional configs needed for
Spark to connect to SqlServer please?
Appreciate if we can get inputs to resolve this error.
Full Stack Trace
Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: This driver is not
configured for integrated authentication. at
com.microsoft.sqlserver.jdbc.SQLServerConnection.terminate(SQLServerConnection.java:1352)
at
com.microsoft.sqlserver.jdbc.SQLServerConnection.sendLogon(SQLServerConnection.java:2329)
at
com.microsoft.sqlserver.jdbc.SQLServerConnection.logon(SQLServerConnection.java:1905)
at
com.microsoft.sqlserver.jdbc.SQLServerConnection.access$000(SQLServerConnection.java:41)
at
com.microsoft.sqlserver.jdbc.SQLServerConnection$LogonCommand.doExecute(SQLServerConnection.java:1893)
at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:4575)
at
com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:1400)
at
com.microsoft.sqlserver.jdbc.SQLServerConnection.connectHelper(SQLServerConnection.java:1045)
at
com.microsoft.sqlserver.jdbc.SQLServerConnection.login(SQLServerConnection.java:817)
at
com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(SQLServerConnection.java:700)
at
com.microsoft.sqlserver.jdbc.SQLServerDriver.connect(SQLServerDriver.java:842)
at
org.apache.spark.sql.execution.datasources.jdbc.connection.BasicConnectionProvider.getConnection(BasicConnectionProvider.scala:49)
at
org.apache.spark.sql.execution.datasources.jdbc.connection.SecureConnectionProvider.getConnection(SecureConnectionProvider.scala:44)
at
org.apache.spark.sql.execution.datasources.jdbc.connection.MSSQLConnectionProvider.org$apache$spark$sql$execution$datasources$jdbc$connection$MSSQLConnectionProvider$$super$getConnection(MSSQLConnectionProvider.scala:69)
at
org.apache.spark.sql.execution.datasources.jdbc.connection.MSSQLConnectionProvider$$anon$1.run(MSSQLConnectionProvider.scala:69)
at
org.apache.spark.sql.execution.datasources.jdbc.connection.MSSQLConnectionProvider$$anon$1.run(MSSQLConnectionProvider.scala:67)
at java.base/java.security.AccessController.doPrivileged(Native
Method) at java.base/javax.security.auth.Subject.doAs(Subject.java:423)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
... 23 more Caused by: java.lang.UnsatisfiedLinkError: no sqljdbc_auth
in java.library.path: [/usr/java/packages/lib, /usr/lib64, /lib64, /lib,
/usr/lib] at
java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2660) at
java.base/java.lang.Runtime.loadLibrary0(Runtime.java:827) at
java.base/java.lang.System.loadLibrary(System.java:1871) at
com.microsoft.sqlserver.jdbc.AuthenticationJNI.<clinit>(AuthenticationJNI.java:32)
at
com.microsoft.sqlserver.jdbc.SQLServerConnection.logon(SQLServerConnection.java:1902)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
