[
https://issues.apache.org/jira/browse/HDDS-11777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17902741#comment-17902741
]
Ethan Rose edited comment on HDDS-11777 at 2/4/25 9:33 PM:
-----------------------------------------------------------
In reference to [this
comment|https://github.com/apache/ozone/pull/7465#discussion_r1852572855], the
intent of this task is common code for filesystem tree iteration, which can be
used by other things like FSO repair, ldb scan, and reverse mapping.
was (Author: erose):
In reference to [this
comment|https://github.com/apache/ozone/pull/7465#discussion_r1852572855], the
intent of this task was actually common code for filesystem tree iteration,
which can be used by other things like FSO repair, ldb scan, and reverse
mapping.
> Create common method for tasks that need to iterate OM namespace in parallel
> ----------------------------------------------------------------------------
>
> Key: HDDS-11777
> URL: https://issues.apache.org/jira/browse/HDDS-11777
> Project: Apache Ozone
> Issue Type: Improvement
> Reporter: Sumit Agrawal
> Priority: Major
>
> Add a utility method that supports iterating the filesystem tree from RocksDB
> in a parallel manner. This has multiple potential use cases, including
> building a container to key mapping from a debug tool, repairing quota
> issues, and namespace processing on Recon snapshot install, like DU and
> container to key mapping.
> This method would be used in cases where we can sacrifice in-order traversal
> of keys to gain parallelization with low memory overhead, using a modified
> DFS with parallel tree splitting for FSO and more simple work splitting for
> OBS. Signature would look something like this:
> {code:java}
> void walkNamespace(DB omDB, String startPath, Consumer<KeyInfo> action, int
> threadCount)
> {code}
> Each use case would specify the action it needs to perform on each key in the
> consumer.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
