[
https://issues.apache.org/jira/browse/HDFS-11926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16038241#comment-16038241
]
Anu Engineer commented on HDFS-11926:
-------------------------------------
[~cheersyang] Thanks for getting to this before me. I was also planning to do
this cleanup, greatly appreciate your work. Some minor comments on the code.
In {{LevelDBStore#getRangeKVs}}
# Should we make MAX_NUM_OF_ENTRIES a config setting instead of hardcoding and
make 1024 the default value ?
# There are couple of subtle race conditions here.
## If startKey is not null, it means that user knows about a key which they
want to seek. so if the db.get(startKey) fails , we could throw an exception.
This is probably a call to continue a listing operation. However, it is quite
possible that startKey may not exist at this point of time. If that is the
case, we can either throw -- so that user knows it is an inconsistent iteration
or have some config or user argument that allows us to continue.
## If there is no startKey (null startkey) that is when we should seek to
first, I am not sure if we need to always seek to first. I am hopeful seek()
works without a seekToFirst before it.
## There is a subtle race condition here, since the seek can race with a delete
and we are not holding any locks while listing. So it is possible for
{{db.get(startkey)}} to return a non-null value, but {{dbIter.seek(startKey)}}
to fail. May be we should take a read lock since we know we will only read at
the maximum of 1024 keys. Otherwise, we might need to operate against a
snapshot.
## I really think we should just take a lock here, {{byte[] preKey =
dbIter.hasPrev() ? dbIter.peekPrev().getKey() : null;}} This statement also has
a race condition, hasPrev() and peekPrev.getKey needs to be atomic or the
getKey can fail, if that is by design, then we need to be explicit about that.
## Same comment on nextKey
## Also we should document that {{getRangeKVs}} should be called with lock
held, if we decide to go that route.
> Ozone: Implement a common helper to return a range of KVs in levelDB
> --------------------------------------------------------------------
>
> Key: HDFS-11926
> URL: https://issues.apache.org/jira/browse/HDFS-11926
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
> Reporter: Weiwei Yang
> Assignee: Weiwei Yang
> Priority: Blocker
> Attachments: HDFS-11926-HDFS-7240.001.patch,
> HDFS-11926-HDFS-7240.002.patch
>
>
> There are quite some *LIST* operations need to get a range of keys or values
> from levelDB, and filter entries with key prefix.
> # HDFS-11782 listKeys
> # HDFS-11779 listBuckets
> # HDFS-11773 listVolumes
> # HDFS-11679 listContainers
> we need to implement a common utility for them.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
