[jira] [Commented] (HADOOP-10604) CryptoFileSystem decorator using xAttrs and KeyProvider
[
https://issues.apache.org/jira/browse/HADOOP-10604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14036042#comment-14036042
]
Uma Maheswara Rao G commented on HADOOP-10604:
--
Please find my further comments on the latest patch.
CryptoFileSystemTestBase.java:
one cryption zone -- one encryption zone
{code}
FSDataOutputStream out = cryptoFS.create(file);
int n = data.length/3;
out.write(data, 0, n);
out.close();
{code}
as practive close them in finally.
{code}
@Test(timeout=12)
{code}
Format off here
CryptoFileSystem.java:
seems like if we configure path{} as zone, we will allow it as empty keyid? we
need to validate and refuse this?
underlyingfsschema - underlyingfsscheme
{code}
for (String zone : zones.split(,)) {
int kstart = zone.indexOf({);
int kend = zone.lastIndexOf(});
if (kstart 0 || kend 0) {
throw new IOException(Key id has not configured for encryption +
zone: + zone);
}
{code}
path can have ',', so splitting with , would create issues right?
My suggestion was like below:
{code}
/** Get authority for CFS */
private String getAuthority(URI underLyingURI) {
String authority = encodeURIForCFSAuthority(underLyingURI);
return authority;
}
private String encodeURIForCFSAuthority(URI underLyingURI) {
String encodedUri = underLyingURI.getScheme() + @;
if (underLyingURI.getHost() != null) {
encodedUri += underLyingURI.getHost();
if (underLyingURI.getPort() != -1) {
encodedUri += :;
encodedUri += underLyingURI.getPort();
}
}
else {
encodedUri += :;
}
return encodedUri;
}
{code}
we can even avoid getAuthority method, because getAuthority and passing uri
gives the impression like, getting authority string from the passed uri
When I try to run the test I am seeing the following
{noformat}
java.io.IOException: Bad configuration of hadoop.security.key.provider.path at
jceks://file\tmp\560bda21-4b91-4acd-965d-12040b689b83/keystore.jceks
at
org.apache.hadoop.crypto.key.KeyProviderFactory.getProviders(KeyProviderFactory.java:70)
at
org.apache.hadoop.crypto.key.KeyShell$Command.getKeyProvider(KeyShell.java:193)
at
org.apache.hadoop.crypto.key.KeyShell$CreateCommand.validate(KeyShell.java:426)
at org.apache.hadoop.crypto.key.KeyShell.run(KeyShell.java:66)
at
org.apache.hadoop.fs.CryptoFileSystemTestBase.initKeyStore(CryptoFileSystemTestBase.java:90)
at
org.apache.hadoop.fs.TestCryptoFileSystemDecoratingLocalFS.init(TestCryptoFileSystemDecoratingLocalFS.java:30)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
at
org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
at
org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
Caused by: java.net.URISyntaxException: Illegal character in authority at index
8: jceks://file\tmp\560bda21-4b91-4acd-965d-12040b689b83/keystore.jceks
at java.net.URI$Parser.fail(Unknown Source)
at java.net.URI$Parser.parseAuthority(Unknown Source)
at java.net.URI$Parser.parseHierarchical(Unknown Source)
at java.net.URI$Parser.parse(Unknown Source)
at java.net.URI.init(Unknown Source)
at
org.apache.hadoop.crypto.key.KeyProviderFactory.getProviders(KeyProviderFactory.java:55)
{noformat}
I tried on windows and seems like \t create issue.
CryptoFileSystem decorator using xAttrs and KeyProvider
---
Key: HADOOP-10604
URL: https://issues.apache.org/jira/browse/HADOOP-10604
Project: Hadoop Common
[jira] [Commented] (HADOOP-10604) CryptoFileSystem decorator using xAttrs and KeyProvider
[
https://issues.apache.org/jira/browse/HADOOP-10604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14031979#comment-14031979
]
Uma Maheswara Rao G commented on HADOOP-10604:
--
Thanks a lot Yi for the patch!
I just went throught the patch. Below are my very initial comments and I will
go through the patch in detail tomorrow.
{code}
if (isNestedEncryptionZone(p)) {
+ throw new IOException(Doen't support nested encryption dirs);
+}
{code}
typo. Doen't - Doesn't ?
Seems like we are not allowing nested encryption zones, but cfs will take
client side configuration, one client would have configured one dir and other
client could configure the sub dir of it. In this case how would we avoid
nested encryption zones to configure from the checks?
decodeCFSURI is doing the some special decoding stuff for replacing @ with ://
etc. But there is no encode method and I think its done directly in
getAuthority, instead can we make like encode and decode method?
Also is it good to document about how to create ezs in other fs? ( I mean to
tell the info about what is the qualification to consider as ez? ex if
underlying fs is hdfs, HdfsAdmin has api to creae EZs)
CryptoFileSystem decorator using xAttrs and KeyProvider
---
Key: HADOOP-10604
URL: https://issues.apache.org/jira/browse/HADOOP-10604
Project: Hadoop Common
Issue Type: Sub-task
Components: fs
Affects Versions: fs-encryption (HADOOP-10150 and HDFS-6134)
Reporter: Alejandro Abdelnur
Assignee: Yi Liu
Fix For: fs-encryption (HADOOP-10150 and HDFS-6134)
Attachments: HADOOP-10604.patch
A FileSystem implementation that wraps an existing filesystem and provides
encryption. It will require the underlying filesystem to support xAttrs. It
will use the KeyProvider API to retrieve encryption keys.
This is mostly the work in the patch HADOOP-10150 minus the crypto streams
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HADOOP-10604) CryptoFileSystem decorator using xAttrs and KeyProvider
[ https://issues.apache.org/jira/browse/HADOOP-10604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14032057#comment-14032057 ] Yi Liu commented on HADOOP-10604: - Thanks Uma for nice review, will respond to you later. CryptoFileSystem decorator using xAttrs and KeyProvider --- Key: HADOOP-10604 URL: https://issues.apache.org/jira/browse/HADOOP-10604 Project: Hadoop Common Issue Type: Sub-task Components: fs Affects Versions: fs-encryption (HADOOP-10150 and HDFS-6134) Reporter: Alejandro Abdelnur Assignee: Yi Liu Fix For: fs-encryption (HADOOP-10150 and HDFS-6134) Attachments: HADOOP-10604.patch A FileSystem implementation that wraps an existing filesystem and provides encryption. It will require the underlying filesystem to support xAttrs. It will use the KeyProvider API to retrieve encryption keys. This is mostly the work in the patch HADOOP-10150 minus the crypto streams -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HADOOP-10604) CryptoFileSystem decorator using xAttrs and KeyProvider
[
https://issues.apache.org/jira/browse/HADOOP-10604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14032059#comment-14032059
]
Yi Liu commented on HADOOP-10604:
-
This JIRA includes tests for {{CryptoFileSystem}} decorating local FS.
In HDFS-6537, there is tests for {{CryptoFileSystem}} decorating HDFS.
{{CryptoFileSystem}} targets other filesystem. But currently other built-in
Hadoop filesystems don't have XAttrs support, so we also use HDFS for test.
CryptoFileSystem decorator using xAttrs and KeyProvider
---
Key: HADOOP-10604
URL: https://issues.apache.org/jira/browse/HADOOP-10604
Project: Hadoop Common
Issue Type: Sub-task
Components: fs
Affects Versions: fs-encryption (HADOOP-10150 and HDFS-6134)
Reporter: Alejandro Abdelnur
Assignee: Yi Liu
Fix For: fs-encryption (HADOOP-10150 and HDFS-6134)
Attachments: HADOOP-10604.patch
A FileSystem implementation that wraps an existing filesystem and provides
encryption. It will require the underlying filesystem to support xAttrs. It
will use the KeyProvider API to retrieve encryption keys.
This is mostly the work in the patch HADOOP-10150 minus the crypto streams
--
This message was sent by Atlassian JIRA
(v6.2#6252)
