GitHub user cmirash opened a pull request:
https://github.com/apache/spark/pull/15594
[SPARK-18061][SQL][Security] Spark Thriftserver needs to create SPNego
principal
## What changes were proposed in this pull request?
Spark Thriftserver when running in HTTP mode with Kerberos enabled gives a
401 authentication error when receiving beeline HTTP request (with end user as
kerberos principal). The similar command works with Hive Thriftserver.
What we find is Hive thriftserver CLI service creates both hive service and
SPNego principal when kerberos is enabled whereas Spark Thriftserver only
creates hive service principal.
CLIService.java
if (UserGroupInformation.isSecurityEnabled()) {
try {
HiveAuthFactory.loginFromKeytab(hiveConf);
this.serviceUGI = Utils.getUGI();
} catch (IOException e) {
throw new ServiceException("Unable to login to kerberos with given
principal/keytab", e);
} catch (LoginException e) {
throw new ServiceException("Unable to login to kerberos with given
principal/keytab", e);
}
// Also try creating a UGI object for the SPNego principal
String principal =
hiveConf.getVar(ConfVars.HIVE_SERVER2_SPNEGO_PRINCIPAL);
String keyTabFile =
hiveConf.getVar(ConfVars.HIVE_SERVER2_SPNEGO_KEYTAB);
if (principal.isEmpty() || keyTabFile.isEmpty()) {
LOG.info("SPNego httpUGI not created, spNegoPrincipal: " +
principal +
", ketabFile: " + keyTabFile);
} else {
try {
this.httpUGI =
HiveAuthFactory.loginFromSpnegoKeytabAndReturnUGI(hiveConf);
LOG.info("SPNego httpUGI successfully created.");
} catch (IOException e) {
LOG.warn("SPNego httpUGI creation failed: ", e);
}
}
}
SparkSQLCLIService.scala
if (UserGroupInformation.isSecurityEnabled) {
try {
HiveAuthFactory.loginFromKeytab(hiveConf)
sparkServiceUGI = Utils.getUGI()
setSuperField(this, "serviceUGI", sparkServiceUGI)
} catch {
case e @ (_: IOException | _: LoginException) =>
throw new ServiceException("Unable to login to kerberos with
given principal/keytab", e)
}
The patch will add missing SPNego principal to Spark Thriftserver.
## How was this patch tested?
Ran manual testing with beeline command through spark against kerberized
cluster.
Ran Spark unit tests for hive, sql and catalyst.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/cmirash/spark spark_SPNego
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/15594.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #15594
----
commit 448f91fd7c8098e35ba0634b724bfc614c0a5342
Author: Chandana Mirashi <[email protected]>
Date: 2016-10-22T00:15:57Z
Changes to create SPNego principal
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
