[
https://issues.apache.org/jira/browse/HDDS-1834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16892790#comment-16892790
]
Lokesh Jain edited comment on HDDS-1834 at 7/25/19 2:14 PM:
------------------------------------------------------------
There are two bugs associated with checkAccess.
# In OzoneFileSystem use cases, for access of a descendant checkAccess of any
ancestor is not done. Currently while accessing a/b/c.txt we do not check the
access for a/ and a/b/ and do a access check only for the path a/b/c.txt
# In HDDS-1481 while doing mkdir, the ancestor directories are not created if
they do not exist. checkAccess method only checks for the key provided and
therefore fails with KEY_NOT_FOUND error. It should do a check for existence of
a directory using getFileStatus.
KeyManagerImpl#checkAccess:1645-1657
{code:java}
OmKeyInfo keyInfo = metadataManager.getKeyTable().get(objectKey);
if (keyInfo == null) {
objectKey = OzoneFSUtils.addTrailingSlashIfNeeded(objectKey);
keyInfo = metadataManager.getKeyTable().get(objectKey);
if(keyInfo == null) {
keyInfo = metadataManager.getOpenKeyTable().get(objectKey);
if (keyInfo == null) {
throw new OMException("Key not found, checkAccess failed. Key:" +
objectKey, KEY_NOT_FOUND);
}
}
}
{code}
Example illustrating the problem 2.
{code:java}
ozone sh key list o3://om/fstest/bucket1/
[ {
"version" : 0,
"md5hash" : null,
"createdOn" : "Thu, 25 Jul 2019 11:26:02 GMT",
"modifiedOn" : "Thu, 25 Jul 2019 11:26:02 GMT",
"size" : 0,
"keyName" : "testdir/deep/",
"type" : null
}, {
"version" : 0,
"md5hash" : null,
"createdOn" : "Thu, 25 Jul 2019 11:26:09 GMT",
"modifiedOn" : "Thu, 01 Jan 1970 00:12:54 GMT",
"size" : 22808,
"keyName" : "testdir/deep/MOVED.TXT",
"type" : null
}, {
"version" : 0,
"md5hash" : null,
"createdOn" : "Thu, 25 Jul 2019 11:26:18 GMT",
"modifiedOn" : "Thu, 01 Jan 1970 00:12:44 GMT",
"size" : 22808,
"keyName" : "testdir/deep/PUTFILE.txt",
"type" : null
} ]
ozone sh key info o3://om/fstest/bucket1/testdir
KEY_NOT_FOUND Key not found, checkAccess failed. Key:/fstest/bucket1/testdir/
{code}
was (Author: ljain):
The problem exists in general for checkAccess. There are two bugs associated
with checkAccess.
# In OzoneFileSystem use cases, for access of a descendant checkAccess of any
ancestor is not done. Currently while accessing a/b/c.txt we do not check the
access for a/ and a/b/ and do a access check only for the path a/b/c.txt
# In HDDS-1481 while doing mkdir, the ancestor directories are not created if
they do not exist. checkAccess method only checks for the key provided and
therefore fails with KEY_NOT_FOUND error. It should do a check for existence of
a directory using getFileStatus.
KeyManagerImpl#checkAccess:1645-1657
{code:java}
OmKeyInfo keyInfo = metadataManager.getKeyTable().get(objectKey);
if (keyInfo == null) {
objectKey = OzoneFSUtils.addTrailingSlashIfNeeded(objectKey);
keyInfo = metadataManager.getKeyTable().get(objectKey);
if(keyInfo == null) {
keyInfo = metadataManager.getOpenKeyTable().get(objectKey);
if (keyInfo == null) {
throw new OMException("Key not found, checkAccess failed. Key:" +
objectKey, KEY_NOT_FOUND);
}
}
}
{code}
Example illustrating the problem 2.
{code:java}
ozone sh key list o3://om/fstest/bucket1/
[ {
"version" : 0,
"md5hash" : null,
"createdOn" : "Thu, 25 Jul 2019 11:26:02 GMT",
"modifiedOn" : "Thu, 25 Jul 2019 11:26:02 GMT",
"size" : 0,
"keyName" : "testdir/deep/",
"type" : null
}, {
"version" : 0,
"md5hash" : null,
"createdOn" : "Thu, 25 Jul 2019 11:26:09 GMT",
"modifiedOn" : "Thu, 01 Jan 1970 00:12:54 GMT",
"size" : 22808,
"keyName" : "testdir/deep/MOVED.TXT",
"type" : null
}, {
"version" : 0,
"md5hash" : null,
"createdOn" : "Thu, 25 Jul 2019 11:26:18 GMT",
"modifiedOn" : "Thu, 01 Jan 1970 00:12:44 GMT",
"size" : 22808,
"keyName" : "testdir/deep/PUTFILE.txt",
"type" : null
} ]
ozone sh key info o3://om/fstest/bucket1/testdir
KEY_NOT_FOUND Key not found, checkAccess failed. Key:/fstest/bucket1/testdir/
{code}
> parent directories not found in secure setup due to ACL check
> -------------------------------------------------------------
>
> Key: HDDS-1834
> URL: https://issues.apache.org/jira/browse/HDDS-1834
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
> Components: Ozone Filesystem
> Reporter: Doroszlai, Attila
> Assignee: Doroszlai, Attila
> Priority: Blocker
>
> ozonesecure-ozonefs acceptance test is failing, because {{ozone fs -mkdir
> -p}} only creates key for the specific directory, not its parents.
> {noformat}
> ozone fs -mkdir -p o3fs://bucket1.fstest/testdir/deep
> {noformat}
> Previous result:
> {noformat:title=https://ci.anzix.net/job/ozone-nightly/176/artifact/hadoop-ozone/dist/target/ozone-0.5.0-SNAPSHOT/compose/result/log.html#s1-s16-t3-k2}
> $ ozone sh key list o3://om/fstest/bucket1 | grep -v WARN | jq -r
> '.[].keyName'
> testdir/
> testdir/deep/
> {noformat}
> Current result:
> {noformat:title=https://ci.anzix.net/job/ozone-nightly/177/artifact/hadoop-ozone/dist/target/ozone-0.5.0-SNAPSHOT/compose/result/log.html#s1-s16-t3-k2}
> $ ozone sh key list o3://om/fstest/bucket1 | grep -v WARN | jq -r
> '.[].keyName'
> testdir/deep/
> {noformat}
> The failure happens on first operation that tries to use {{testdir/}}
> directly:
> {noformat}
> $ ozone fs -touch o3fs://bucket1.fstest/testdir/TOUCHFILE.txt
> ls: `o3fs://bucket1.fstest/testdir': No such file or directory
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
