This is an automatically generated e-mail. To reply, visit:

(Updated Sept. 7, 2016, 4:13 a.m.)

Review request for Ambari, Di Li, DIPAYAN BHOWMICK, and Pallav Kulshreshtha.


added test case

Bugs: AMBARI-18071

Repository: ambari


When HDFS is configured with Encryption Zones, Files View to browse files will 
give "No KeyProvider" error.

Steps to reproduce this issue:
1. Configure an encrypted zone in HDFS (Transparent Data Encryption). I have 
used Hadoop's KMS (installed tar manually).
2. Create a Files View instance and provide a user/group with the privilege to 
use the instance.
3. Log into the Ambari console as the user with the Files View permission.
4. Open the Files View instance.
5. Go to the folder which is configured as an encrypted zone.
6. Try to open an existing file in this folder.
7. This throws an error - java.io.IOException: No KeyProvider is configured, 
cannot access an encrypted file. 
8. When trying through the shell, opening this file works.

This happens because Files View doesn't have enough configuration set to browse 
secured zone. Files view doesn't even provide an option to add this 
configuration. This is why we see errors "No KeyProvider is configured, cannot 
access an encrypted file". 
To work around this, you could download client configuration from HDFS service 
tab, and copy the core-site.xml and hdfs-site.xml files to 
/etc/ambari-server/conf, then restart ambari-server. After this, the user is 
able to open the file in the encrypted zone. Basically, the property 
"dfs.encryption.key.provider.uri" which provides details of the KeyProvider, is 
obtained from the hdfs-site.xml present in the /etc/ambari-server/conf folder.

Diffs (updated)


Diff: https://reviews.apache.org/r/51578/diff/


The error of "No KeyProvider is configured" is seen only for those cases when 
the HDFS uses DistributedFileSystem for its communication. When HDFS uses 
WebHDFSFileSystem for communication, this error is not seen and the Ambari View 
instance is able to open the files in the encrypted zones.

Why Ambari Views use either Distributed or WebHDFS file systems is explained 
Ambari views can be created using one of the 3 modes of configuration:
1. Local cluster
2. Remote cluster
3. Custom configuration (no cluster is associated here).

The HDFS works through abstraction. For Ambari Views, the actual file system 
used during execution depends on whether the view instance was created using a 
Local/Remote cluster or using Custom configuration. For instances created using 
Local/Remote cluster, HDFS uses Distributed File System and for instances 
created using Custom configuration, HDFS uses WebHDFSFileSystem.
WebHDFSFileSystem is an integrated part of the HDFS ecosystem. It is aware of 
all the HDFS configuration. For this reason, when a KMS is configured in HDFS, 
WebHDFSFileSystem is aware of the KeyProvider and no special config mapping is 
needed. Thus, even the view instance created using Custom configuration doesn't 
need any special configuration and can talk to the Encryption Zones 

However, for view instances created using Local/Remote cluster configuration, 
HDFS uses the Distributed FileSystem. This Distributed FileSystem works as an 
HDFS client and hence, is not fully aware of all the HDFS configuration. We 
need to explicitly provide HDFS properties like 
"dfs.encryption.key.provider.uri" to these ambari view instances to provide 
details of the KeyProvider. The proposed fix helps in providing this property 
value to the view as follows.

The proposed fix (attached as "AMBARI-18071.patch") checks if the current view 
instance configuration has any cluster associated in its context. If there is 
an associated cluster then the instance has a Local/Remote cluster 
configuration and needs to be provided with the HDFS KeyProvider information. 
Otherwise, the WebHDFSFileSystem will take care of the KeyProvider if KMS is 

To provide the property information, the parseProperties() in 
ConfigurationBuilder.java looked best as we also set the defaultFS property 
here. If a cluster is associated with the context, and if the property 
"dfs.encryption.key.provider.uri" is not null, then this property is set in the 
Configuration object and thus made available to Distributed file system of HDFS.
The Ambari VIew instance works successfully with both Local and Remote 

One more point to note in the configuration aspect is the addition of proxyuser 
to the kms-site.xml for the ambari-server daemon. Without this proxyuser even 
the custom configuration will not work. (I had installed hadoop's KMS on the 
ambari-server manually)

I have done manual testing.
Log in as a user who is allowed to access encrypted zones.
Go to the Ambari View instance (with Local/Remote cluster configuration).
Open the encrypted zone folder and open an existing file in this directory.
The user can successfully preview the file without the "No KeyProvider 
configured" error.


Keta Patel

Reply via email to