[
https://issues.apache.org/jira/browse/HDDS-1984?focusedWorklogId=324699&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-324699
]
ASF GitHub Bot logged work on HDDS-1984:
----------------------------------------
Author: ASF GitHub Bot
Created on: 07/Oct/19 23:39
Start Date: 07/Oct/19 23:39
Worklog Time Spent: 10m
Work Description: bharatviswa504 commented on pull request #1555:
HDDS-1984. Fix listBucket API.
URL: https://github.com/apache/hadoop/pull/1555#discussion_r332282739
##########
File path:
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OmMetadataManagerImpl.java
##########
@@ -618,23 +618,31 @@ public boolean isBucketEmpty(String volume, String
bucket)
}
int currentCount = 0;
- try (TableIterator<String, ? extends KeyValue<String, OmBucketInfo>>
- bucketIter = bucketTable.iterator()) {
- KeyValue<String, OmBucketInfo> kv = bucketIter.seek(startKey);
- while (currentCount < maxNumOfBuckets && bucketIter.hasNext()) {
- kv = bucketIter.next();
- // Skip the Start Bucket if needed.
- if (kv != null && skipStartKey &&
- kv.getKey().equals(startKey)) {
+
+ // For Bucket it is full cache, so we can just iterate in-memory table
+ // cache.
+ Iterator<Map.Entry<CacheKey<String>, CacheValue<OmBucketInfo>>> iterator =
Review comment:
There is a maxCount, when we reach the count, we will return immediately, in
that case, we shall not iterate all volumes in a bucket.
BucketTable cache is just a concurrentHashMap of all buckets in OM.
So let's take an example: As it is ConcurrentSkipListMap, it has items
sorted based on the key.
We have entries in bucket table cache like below: (For brevity, removed
bucketInfo structures which are the values for the key.)
/vol/buck1
/vol/buck2
/vol/buck3
/vol2/bucket2
/vol2/bucket3
/vol2/bucket4
When we want to list buckets of /vol, and in each return only 1
entry(maximum count), we return /vol/buck1 and then immediately return, next
time listBuckets is called, startkey will be /vol/buck1,1, we return
/vol/buck2, (To return this we iterate 2 entries), like that so on.
But when we want to list all buckets is /vol2, we will iterate the entries
from start, and reach to /vol2 in the cache and once the maximum count is
reached we return from there.
So, to answer your question not every time we iterate entire cache map. (But
some times we iterate and skip them, as shown with case of vol2 bucket list)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 324699)
Time Spent: 1h 40m (was: 1.5h)
> Fix listBucket API
> ------------------
>
> Key: HDDS-1984
> URL: https://issues.apache.org/jira/browse/HDDS-1984
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
> Reporter: Bharat Viswanadham
> Assignee: Bharat Viswanadham
> Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> This Jira is to fix listBucket API in HA code path.
> In HA, we have an in-memory cache, where we put the result to in-memory cache
> and return the response, later it will be picked by double buffer thread and
> it will flush to disk. So, now when do listBuckets, it should use both
> in-memory cache and rocksdb bucket table to list buckets in a volume.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
