[ https://issues.apache.org/jira/browse/HDDS-1834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16892790#comment-16892790 ]
Lokesh Jain edited comment on HDDS-1834 at 7/25/19 2:14 PM: ------------------------------------------------------------ There are two bugs associated with checkAccess. # In OzoneFileSystem use cases, for access of a descendant checkAccess of any ancestor is not done. Currently while accessing a/b/c.txt we do not check the access for a/ and a/b/ and do a access check only for the path a/b/c.txt # In HDDS-1481 while doing mkdir, the ancestor directories are not created if they do not exist. checkAccess method only checks for the key provided and therefore fails with KEY_NOT_FOUND error. It should do a check for existence of a directory using getFileStatus. KeyManagerImpl#checkAccess:1645-1657 {code:java} OmKeyInfo keyInfo = metadataManager.getKeyTable().get(objectKey); if (keyInfo == null) { objectKey = OzoneFSUtils.addTrailingSlashIfNeeded(objectKey); keyInfo = metadataManager.getKeyTable().get(objectKey); if(keyInfo == null) { keyInfo = metadataManager.getOpenKeyTable().get(objectKey); if (keyInfo == null) { throw new OMException("Key not found, checkAccess failed. Key:" + objectKey, KEY_NOT_FOUND); } } } {code} Example illustrating the problem 2. {code:java} ozone sh key list o3://om/fstest/bucket1/ [ { "version" : 0, "md5hash" : null, "createdOn" : "Thu, 25 Jul 2019 11:26:02 GMT", "modifiedOn" : "Thu, 25 Jul 2019 11:26:02 GMT", "size" : 0, "keyName" : "testdir/deep/", "type" : null }, { "version" : 0, "md5hash" : null, "createdOn" : "Thu, 25 Jul 2019 11:26:09 GMT", "modifiedOn" : "Thu, 01 Jan 1970 00:12:54 GMT", "size" : 22808, "keyName" : "testdir/deep/MOVED.TXT", "type" : null }, { "version" : 0, "md5hash" : null, "createdOn" : "Thu, 25 Jul 2019 11:26:18 GMT", "modifiedOn" : "Thu, 01 Jan 1970 00:12:44 GMT", "size" : 22808, "keyName" : "testdir/deep/PUTFILE.txt", "type" : null } ] ozone sh key info o3://om/fstest/bucket1/testdir KEY_NOT_FOUND Key not found, checkAccess failed. Key:/fstest/bucket1/testdir/ {code} was (Author: ljain): The problem exists in general for checkAccess. There are two bugs associated with checkAccess. # In OzoneFileSystem use cases, for access of a descendant checkAccess of any ancestor is not done. Currently while accessing a/b/c.txt we do not check the access for a/ and a/b/ and do a access check only for the path a/b/c.txt # In HDDS-1481 while doing mkdir, the ancestor directories are not created if they do not exist. checkAccess method only checks for the key provided and therefore fails with KEY_NOT_FOUND error. It should do a check for existence of a directory using getFileStatus. KeyManagerImpl#checkAccess:1645-1657 {code:java} OmKeyInfo keyInfo = metadataManager.getKeyTable().get(objectKey); if (keyInfo == null) { objectKey = OzoneFSUtils.addTrailingSlashIfNeeded(objectKey); keyInfo = metadataManager.getKeyTable().get(objectKey); if(keyInfo == null) { keyInfo = metadataManager.getOpenKeyTable().get(objectKey); if (keyInfo == null) { throw new OMException("Key not found, checkAccess failed. Key:" + objectKey, KEY_NOT_FOUND); } } } {code} Example illustrating the problem 2. {code:java} ozone sh key list o3://om/fstest/bucket1/ [ { "version" : 0, "md5hash" : null, "createdOn" : "Thu, 25 Jul 2019 11:26:02 GMT", "modifiedOn" : "Thu, 25 Jul 2019 11:26:02 GMT", "size" : 0, "keyName" : "testdir/deep/", "type" : null }, { "version" : 0, "md5hash" : null, "createdOn" : "Thu, 25 Jul 2019 11:26:09 GMT", "modifiedOn" : "Thu, 01 Jan 1970 00:12:54 GMT", "size" : 22808, "keyName" : "testdir/deep/MOVED.TXT", "type" : null }, { "version" : 0, "md5hash" : null, "createdOn" : "Thu, 25 Jul 2019 11:26:18 GMT", "modifiedOn" : "Thu, 01 Jan 1970 00:12:44 GMT", "size" : 22808, "keyName" : "testdir/deep/PUTFILE.txt", "type" : null } ] ozone sh key info o3://om/fstest/bucket1/testdir KEY_NOT_FOUND Key not found, checkAccess failed. Key:/fstest/bucket1/testdir/ {code} > parent directories not found in secure setup due to ACL check > ------------------------------------------------------------- > > Key: HDDS-1834 > URL: https://issues.apache.org/jira/browse/HDDS-1834 > Project: Hadoop Distributed Data Store > Issue Type: Bug > Components: Ozone Filesystem > Reporter: Doroszlai, Attila > Assignee: Doroszlai, Attila > Priority: Blocker > > ozonesecure-ozonefs acceptance test is failing, because {{ozone fs -mkdir > -p}} only creates key for the specific directory, not its parents. > {noformat} > ozone fs -mkdir -p o3fs://bucket1.fstest/testdir/deep > {noformat} > Previous result: > {noformat:title=https://ci.anzix.net/job/ozone-nightly/176/artifact/hadoop-ozone/dist/target/ozone-0.5.0-SNAPSHOT/compose/result/log.html#s1-s16-t3-k2} > $ ozone sh key list o3://om/fstest/bucket1 | grep -v WARN | jq -r > '.[].keyName' > testdir/ > testdir/deep/ > {noformat} > Current result: > {noformat:title=https://ci.anzix.net/job/ozone-nightly/177/artifact/hadoop-ozone/dist/target/ozone-0.5.0-SNAPSHOT/compose/result/log.html#s1-s16-t3-k2} > $ ozone sh key list o3://om/fstest/bucket1 | grep -v WARN | jq -r > '.[].keyName' > testdir/deep/ > {noformat} > The failure happens on first operation that tries to use {{testdir/}} > directly: > {noformat} > $ ozone fs -touch o3fs://bucket1.fstest/testdir/TOUCHFILE.txt > ls: `o3fs://bucket1.fstest/testdir': No such file or directory > {noformat} -- This message was sent by Atlassian JIRA (v7.6.14#76016) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org