[
https://issues.apache.org/jira/browse/HDFS-13281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16419057#comment-16419057
]
Rushabh S Shah commented on HDFS-13281:
---------------------------------------
Thanks [~xiaochen] for the comment.
bq. so this patch only applies if step #1 does a startFile with /.reserved/raw
Yes.
bq. in which case there will be no step #3
Didn't get you. can you please elaborate.
bq. And step#5 also only applies to files created with /.reserved/raw.
Yes. but only if namenode returned {{FeInfo}} to client in response to
{{createFile}} call.
WebhdfsFileSystem won't be able to know that path is {{/.reserved/raw}}.
WebhdfsFileSystem will issue {{createFile}} call (without /.reserved/raw) path.
Namenode will check whether path is in EZ and whether client indicated EZ
support via header.
If both the conditions are met, then namenode will return {{FeInfo}} and it
will prepend the redirect path with {{/.reserved/raw}}.
Client never reads the redirect path so client will issue {{setXAttr}} just
based on whether namenode returned {{FEInfo}}.
bq. Could you rebase the patch? Doesn't apply to trunk.
Done.
bq. The test doesn't verify that there was no intermediate EDEK consumed and
set on the file.
Currently on create call, namenode will generate edek and sets the xattr
{{raw.hdfs.crypto.file.encryption.info}} if the path is in an EZ irrespective
whether path is {{/.reserved/raw}}.
In test case, I am creating a file with {{/.reserved/raw}} mimicking the way
datanode will issue {{createFile}} to namenode.
In test, I am checking whether {{raw.hdfs.crypto.file.encryption.info}} attr is
set or not.
If it is set that means namenode generated edek and if not, then it means
namenode didn't generate edek.
bq. Can you add a distcp test as you mentioned before?
As I mentioned before, distcp is just an example. The problem lies in namenode
generating edek even though the path is {{/.reserved/raw}}. So I think the test
case is sufficient to prove that namenode didn't generate edek if path is
{{/.reserved/raw}}.
> Namenode#createFile should be /.reserved/raw/ aware.
> ----------------------------------------------------
>
> Key: HDFS-13281
> URL: https://issues.apache.org/jira/browse/HDFS-13281
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: encryption
> Affects Versions: 2.8.3
> Reporter: Rushabh S Shah
> Assignee: Rushabh S Shah
> Priority: Critical
> Attachments: HDFS-13281.001.patch, HDFS-13281.002.patch
>
>
> If I want to write to /.reserved/raw/<dir> and if that directory happens to
> be in EZ, then namenode *should not* create edek and just copy the raw bytes
> from the source.
> Namenode#startFileInt should be /.reserved/raw/ aware.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
