> On Sept. 8, 2016, 2:41 p.m., Di Li wrote: > > Ship It!
Thank you Di! - Keta ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51578/#review148187 ----------------------------------------------------------- On Sept. 7, 2016, 4:57 p.m., Keta Patel wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/51578/ > ----------------------------------------------------------- > > (Updated Sept. 7, 2016, 4:57 p.m.) > > > Review request for Ambari, Di Li, DIPAYAN BHOWMICK, and Pallav Kulshreshtha. > > > Bugs: AMBARI-18071 > https://issues.apache.org/jira/browse/AMBARI-18071 > > > Repository: ambari > > > Description > ------- > > When HDFS is configured with Encryption Zones, Files View to browse files > will give "No KeyProvider" error. > > Steps to reproduce this issue: > 1. Configure an encrypted zone in HDFS (Transparent Data Encryption). I have > used Hadoop's KMS (installed tar manually). > 2. Create a Files View instance and provide a user/group with the privilege > to use the instance. > 3. Log into the Ambari console as the user with the Files View permission. > 4. Open the Files View instance. > 5. Go to the folder which is configured as an encrypted zone. > 6. Try to open an existing file in this folder. > 7. This throws an error - java.io.IOException: No KeyProvider is configured, > cannot access an encrypted file. > 8. When trying through the shell, opening this file works. > > This happens because Files View doesn't have enough configuration set to > browse secured zone. Files view doesn't even provide an option to add this > configuration. This is why we see errors "No KeyProvider is configured, > cannot access an encrypted file". > To work around this, you could download client configuration from HDFS > service tab, and copy the core-site.xml and hdfs-site.xml files to > /etc/ambari-server/conf, then restart ambari-server. After this, the user is > able to open the file in the encrypted zone. Basically, the property > "dfs.encryption.key.provider.uri" which provides details of the KeyProvider, > is obtained from the hdfs-site.xml present in the /etc/ambari-server/conf > folder. > > > Diffs > ----- > > > contrib/views/utils/src/main/java/org/apache/ambari/view/utils/hdfs/ConfigurationBuilder.java > d505e9a > > contrib/views/utils/src/test/java/org/apache/ambari/view/utils/hdfs/ConfigurationBuilderTest.java > ed3025b > > Diff: https://reviews.apache.org/r/51578/diff/ > > > Testing > ------- > > **CAUSE:** > ========== > The error of "No KeyProvider is configured" is seen only for those cases when > the HDFS uses DistributedFileSystem for its communication. When HDFS uses > WebHDFSFileSystem for communication, this error is not seen and the Ambari > View instance is able to open the files in the encrypted zones. > > Why Ambari Views use either Distributed or WebHDFS file systems is explained > below: > Ambari views can be created using one of the 3 modes of configuration: > 1. Local cluster > 2. Remote cluster > 3. Custom configuration (no cluster is associated here). > > The HDFS works through abstraction. For Ambari Views, the actual file system > used during execution depends on whether the view instance was created using > a Local/Remote cluster or using Custom configuration. For instances created > using Local/Remote cluster, HDFS uses Distributed File System and for > instances created using Custom configuration, HDFS uses WebHDFSFileSystem. > WebHDFSFileSystem is an integrated part of the HDFS ecosystem. It is aware of > all the HDFS configuration. For this reason, when a KMS is configured in > HDFS, WebHDFSFileSystem is aware of the KeyProvider and no special config > mapping is needed. Thus, even the view instance created using Custom > configuration doesn't need any special configuration and can talk to the > Encryption Zones successfully. > > However, for view instances created using Local/Remote cluster configuration, > HDFS uses the Distributed FileSystem. This Distributed FileSystem works as an > HDFS client and hence, is not fully aware of all the HDFS configuration. We > need to explicitly provide HDFS properties like > "dfs.encryption.key.provider.uri" to these ambari view instances to provide > details of the KeyProvider. The proposed fix helps in providing this property > value to the view as follows. > > **FIX:** > ======== > The proposed fix (attached as "AMBARI-18071.patch") checks if the current > view instance configuration has any cluster associated in its context. If > there is an associated cluster then the instance has a Local/Remote cluster > configuration and needs to be provided with the HDFS KeyProvider information. > Otherwise, the WebHDFSFileSystem will take care of the KeyProvider if KMS is > configured. > > To provide the property information, the parseProperties() in > ConfigurationBuilder.java looked best as we also set the defaultFS property > here. If a cluster is associated with the context, and if the property > "dfs.encryption.key.provider.uri" is not null, then this property is set in > the Configuration object and thus made available to Distributed file system > of HDFS. > The Ambari VIew instance works successfully with both Local and Remote > configurations. > > One more point to note in the configuration aspect is the addition of > proxyuser to the kms-site.xml for the ambari-server daemon. Without this > proxyuser even the custom configuration will not work. (I had installed > hadoop's KMS on the ambari-server manually) > > > **TESTING:** > ============ > I have done manual testing. > Log in as a user who is allowed to access encrypted zones. > Go to the Ambari View instance (with Local/Remote cluster configuration). > Open the encrypted zone folder and open an existing file in this directory. > The user can successfully preview the file without the "No KeyProvider > configured" error. > > > File Attachments > ---------------- > > AMBARI-18071-Sep6.patch > > https://reviews.apache.org/media/uploaded/files/2016/09/07/ca2655a2-7ad1-4af8-abee-b952b4040e4b__AMBARI-18071-Sep6.patch > > > Thanks, > > Keta Patel > >
