[
https://issues.apache.org/jira/browse/HDDS-11777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ethan Rose updated HDDS-11777:
------------------------------
Description:
Add a utility method that supports iterating the filesystem tree from RocksDB
in a parallel manner. This has multiple potential use cases, including building
a container to key mapping from a debug tool, repairing quota issues, and
namespace processing on Recon snapshot install, like DU and container to key
mapping.
This method would be used in cases where we can sacrifice in-order traversal of
keys to gain parallelization with low memory overhead, using a modified DFS
with parallel tree splitting for FSO and more simple work splitting for OBS.
Signature would look something like this:
{code:java}
void walkNamespace(DB omDB, String startPath, Consumer<KeyInfo> action, int
threadCount)
{code}
Each use case would specify the action it needs to perform on each key in the
consumer.
was:
new db added as part of [https://github.com/apache/ozone/pull/7465]
similar logic will be useful in other cases, so this code can be generic to
common place.
> Common code for reverse mapping db
> ----------------------------------
>
> Key: HDDS-11777
> URL: https://issues.apache.org/jira/browse/HDDS-11777
> Project: Apache Ozone
> Issue Type: Improvement
> Reporter: Sumit Agrawal
> Priority: Major
>
> Add a utility method that supports iterating the filesystem tree from RocksDB
> in a parallel manner. This has multiple potential use cases, including
> building a container to key mapping from a debug tool, repairing quota
> issues, and namespace processing on Recon snapshot install, like DU and
> container to key mapping.
> This method would be used in cases where we can sacrifice in-order traversal
> of keys to gain parallelization with low memory overhead, using a modified
> DFS with parallel tree splitting for FSO and more simple work splitting for
> OBS. Signature would look something like this:
> {code:java}
> void walkNamespace(DB omDB, String startPath, Consumer<KeyInfo> action, int
> threadCount)
> {code}
> Each use case would specify the action it needs to perform on each key in the
> consumer.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
