Vihang Karajgaonkar created HIVE-14822:
------------------------------------------
Summary: Add support for credential provider for jobs launched
from Hiveserver2
Key: HIVE-14822
URL: https://issues.apache.org/jira/browse/HIVE-14822
Project: Hive
Issue Type: Bug
Components: HiveServer2
Reporter: Vihang Karajgaonkar
Assignee: Vihang Karajgaonkar
When using encrypted passwords via the Hadoop Credential Provider, HiveServer2
currently does not correctly forward enough information to the job
configuration for jobs to read those secrets. If your job needs to access any
secrets, like S3 credentials, then there's no convenient and secure way to
configure this today.
You could specify the decryption key in files like mapred-site.xml that
HiveServer2 uses, but this would place the encryption password on local disk in
plaintext, which can be a security concern.
To solve this problem, HiveServer2 should modify job configuration to include
the environment variable settings needed to decrypt the passwords.
Specifically, it will need to modify:
* For MR2 jobs:
** yarn.app.mapreduce.am.admin.user.env
** mapreduce.admin.user.env
* For Spark jobs:
** spark.yarn.appMasterEnv.HADOOP_CREDSTORE_PASSWORD
** spark.executorEnv.HADOOP_CREDSTORE_PASSWORD
HiveServer2 can get the decryption password from its own environment, the same
way it does for its own credential provider store today.
Additionally, it can be desirable for HiveServer2 to have a separate encrypted
password file than what is used by the job. HiveServer2 may have secrets that
the job should not have, such as the metastore database password or the
password to decrypt its private SSL certificate. It is also best practices to
have separate passwords on separate files. To facilitate this, Hive will also
accept:
* A configuration for a path to a credential store to use for jobs. This should
already be uploaded in HDFS. (hive.server2.job.keystore.location or a better
name) If this is not specified, then HS2 will simply use the value of
hadoop.security.credential.provider.path.
* An environment variable for the password to decrypt the credential store
(HIVE_JOB_KEYSTORE_PASSWORD or better). If this is not specified, then HS2 will
simply use the standard environment variable for decrypting the Hadoop
Credential Provider.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
