smengcl commented on a change in pull request #108: HDDS-1987. Fix listStatus
API
URL: https://github.com/apache/hadoop-ozone/pull/108#discussion_r350198230
##########
File path:
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/KeyManagerImpl.java
##########
@@ -1884,57 +1899,120 @@ public OmKeyInfo lookupFile(OmKeyArgs args, String
clientAddress)
startKey = OzoneFSUtils.addTrailingSlashIfNeeded(keyName);
}
+ int countEntries = 0;
+ Table keyTable = metadataManager.getKeyTable();
+ Iterator<Map.Entry<CacheKey<String>, CacheValue<OmKeyInfo>>>
+ cacheIter = keyTable.cacheIterator();
+ String startCacheKey = OZONE_URI_DELIMITER + volumeName +
+ OZONE_URI_DELIMITER + bucketName + OZONE_URI_DELIMITER +
+ ((startKey.equals(OZONE_URI_DELIMITER)) ? "" : startKey);
+ // Note: eliminating the case where startCacheKey could end with '//'
+
+ // First, find key in TableCache
+ while (cacheIter.hasNext() && numEntries - countEntries > 0) {
+ Map.Entry<CacheKey<String>, CacheValue<OmKeyInfo>> entry =
+ cacheIter.next();
+ String cacheKey = entry.getKey().getCacheKey();
+ OmKeyInfo cacheOmKeyInfo = entry.getValue().getCacheValue();
+ // cacheOmKeyInfo is null if an entry is deleted in cache
+ if (cacheOmKeyInfo != null) {
+ if (cacheKey.startsWith(startCacheKey) &&
+ cacheKey.compareTo(startCacheKey) >= 0) {
+ if (!recursive) {
+ String remainingKey = StringUtils.stripEnd(cacheKey.substring(
+ startCacheKey.length()), OZONE_URI_DELIMITER);
+ // For non-recursive, the remaining part of key can't have '/'
+ if (remainingKey.contains(OZONE_URI_DELIMITER)) {
+ continue;
+ }
+ }
+ OzoneFileStatus fileStatus = new OzoneFileStatus(
+ cacheOmKeyInfo, scmBlockSize, !OzoneFSUtils.isFile(cacheKey));
+ cacheKeyMap.put(cacheKey, fileStatus);
+ countEntries++;
+ }
+ } else {
+ deletedKeySet.add(cacheKey);
+ }
+ }
+
+ // Then, find key in DB
String seekKeyInDb =
metadataManager.getOzoneKey(volumeName, bucketName, startKey);
String keyInDb = OzoneFSUtils.addTrailingSlashIfNeeded(
metadataManager.getOzoneKey(volumeName, bucketName, keyName));
TableIterator<String, ? extends Table.KeyValue<String, OmKeyInfo>>
- iterator = metadataManager.getKeyTable().iterator();
+ iterator = keyTable.iterator();
iterator.seek(seekKeyInDb);
- if (!iterator.hasNext()) {
- return Collections.emptyList();
- }
-
- if (iterator.key().equals(keyInDb)) {
- // skip the key which needs to be listed
- iterator.next();
- }
-
- while (iterator.hasNext() && numEntries - fileStatusList.size() > 0) {
- String entryInDb = iterator.key();
- OmKeyInfo value = iterator.value().getValue();
- if (entryInDb.startsWith(keyInDb)) {
- String entryKeyName = value.getKeyName();
- if (recursive) {
- // for recursive list all the entries
- fileStatusList.add(new OzoneFileStatus(value, scmBlockSize,
- !OzoneFSUtils.isFile(entryKeyName)));
- iterator.next();
- } else {
- // get the child of the directory to list from the entry. For
- // example if directory to list is /a and entry is /a/b/c where
- // c is a file. The immediate child is b which is a directory. c
- // should not be listed as child of a.
- String immediateChild = OzoneFSUtils
- .getImmediateChild(entryKeyName, keyName);
- boolean isFile = OzoneFSUtils.isFile(immediateChild);
- if (isFile) {
- fileStatusList
- .add(new OzoneFileStatus(value, scmBlockSize, !isFile));
+ if (iterator.hasNext()) {
+ if (iterator.key().equals(keyInDb)) {
Review comment:
Do you mean the `iterator.key().equals(keyInDb)` check? This is just to skip
listing the directory itself.
e.g. `-ls /vol1/buc1/dir1/` should show files and dirs under `dir1` but not
`dir1` itself.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
