[
https://issues.apache.org/jira/browse/HADOOP-14333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15977595#comment-15977595
]
Rushabh S Shah commented on HADOOP-14333:
-----------------------------------------
If you read back [~daryn]'s
[comment|https://issues.apache.org/jira/browse/HADOOP-14104?focusedCommentId=15924596&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15924596]
in HADOOP-14104, we discussed the exact same thing and came to decision that
since DFSClient is a private interface, we are allowed to change the signature
of isHDFSEncryptionEnabled and are allowed to throw IOException.
Since getTrashRoot was doing the wrong thing before HADOOP-14104 (Refer to
https://github.com/apache/hadoop/blob/branch-2.8/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java#L2498),
we decided to do the same thing. At least that is limited to shell commands
not jobs.
bq. Maybe we can temporarily change the behavior here to issue a warning
message and return false when exception is throw.
On this, I still don't think to return false in case of any exception.
If we want to temporarily fix hive use case, we can call
{{DFSClient#getKeyProviderUri}} from {{DistributedFileSystem#getTrashRoot}}.
This means {{DFSClient#isHDFSEncryptionEnabled}} is just called from hive code.
Do something like this in {{DFSClient#isHDFSEncryptionEnabled}}
{code:title=DFSClient.java|borderStyle=solid}
/**
* Probe for encryption enabled on this filesystem.
* @return true if encryption is enabled
*/
public boolean isHDFSEncryptionEnabled() {
try {
URI keyProviderUri = getKeyProviderUri();
} catch(IOException ioe) {
return false;
}
return keyProviderUri !=null
{code}
Probably add a comment on isHDFSEncryptionEnabled saying that this is here only
to unblock hive and no-one should use this.
> New exception thrown by (private) DFSClient API isHDFSEncryptionEnabled broke
> hacky hive code
> ----------------------------------------------------------------------------------------------
>
> Key: HADOOP-14333
> URL: https://issues.apache.org/jira/browse/HADOOP-14333
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 2.8.1, 3.0.0-alpha3
> Reporter: Yongjun Zhang
> Assignee: Yongjun Zhang
>
> Though Hive should be fixed not to access DFSClient which is private to
> HADOOP, removing the throws added by HADOOP-14104 is a quicker solution to
> unblock hive.
> Hive code
> {code}
> private boolean isEncryptionEnabled(DFSClient client, Configuration conf) {
> try {
> DFSClient.class.getMethod("isHDFSEncryptionEnabled");
> } catch (NoSuchMethodException e) {
> // the method is available since Hadoop-2.7.1
> // if we run with an older Hadoop, check this ourselves
> return !conf.getTrimmed(DFSConfigKeys.DFS_ENCRYPTION_KEY_PROVIDER_URI,
> "").isEmpty();
> }
> return client.isHDFSEncryptionEnabled();
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
