GitHub user yolandagao opened a pull request:
https://github.com/apache/spark/pull/9272
[SPARK-10181][SQL] Do kerberos login for credentials during hive client
initialization
On driver process start up, UserGroupInformation.loginUserFromKeytab is
called with the principal and keytab passed in, and therefore static var
UserGroupInfomation,loginUser is set to that principal with kerberos
credentials saved in its private credential set, and all threads within the
driver process are supposed to see and use this login credentials to
authenticate with Hive and Hadoop. However, because of IsolatedClientLoader,
UserGroupInformation class is not shared for hive metastore clients, and
instead it is loaded separately and of course not able to see the prepared
kerberos login credentials in the main thread.
The first proposed fix would cause other classloader conflict errors, and
is not an appropriate solution. This new change does kerberos login during hive
client initialization, which will make credentials ready for the particular
hive client instance.
@yhuai Please take a look and let me know. If you are not the right person
to talk to, could you point me to someone responsible for this?
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/yolandagao/spark master
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/9272.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #9272
----
commit 59253b3bc731338df87301ed3f2a2a8432af8a8b
Author: Yu Gao <[email protected]>
Date: 2015-10-21T01:23:24Z
SPARK-10181 UserGroupInformation class needs to be shared for login
credentials
commit 3e79e9476ad8f922fe7ff0dc5c72767754fd0ff2
Author: Yu Gao <[email protected]>
Date: 2015-10-22T00:05:36Z
Merge remote branch 'upstream/master'
commit 9efa819b8ef1b7d0072de1119da2a0329484c16b
Author: Yu Gao <[email protected]>
Date: 2015-10-24T02:17:19Z
Revert "SPARK-10181 UserGroupInformation class needs to be shared for login
credentials"
This reverts commit 59253b3bc731338df87301ed3f2a2a8432af8a8b.
commit 793458590e5b1fe18e34d793ea31cce2b0576c4b
Author: Yu Gao <[email protected]>
Date: 2015-10-25T01:27:44Z
Merge remote branch 'upstream/master'
commit 30a7d0529ded31c29b5c60e62cb80f7648606e0c
Author: Yu Gao <[email protected]>
Date: 2015-10-25T19:54:01Z
SPARK-10181 Do kerberos login for credentials in hive client initialization
commit 836bb8bada29bf7366d6d04135bfc88cdecd96c8
Author: Yu Gao <[email protected]>
Date: 2015-10-25T19:56:11Z
Merge remote branch 'upstream/master'
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
