[
https://issues.apache.org/jira/browse/HDDS-5332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yuan Gu updated HDDS-5332:
--------------------------
Description:
Table schema:
{code:java}
objectId --> class NSSummary
{code}
New class NSSummary should at least include these fields:
1. Total number of files *directly*[1] under this directory,
2. Total size of files directly under this directory,
3. File Size Buckets of files directly under this directory.
Note: [1] *directly* means not counting files under any subdirectories, for
now. We could further optimize the speed later by trading more writes
(propagating the statistics any layers deep upwards) with speed. This is for
reducing potential write-amplification for now. The stats could be extended
into arbitrary layers deep later.
[06/21] Renamed ReconContainerDBProvider to ReconDBProvider
[06/21] Expanded the scope of this issue by designing a service provider
interface for namespace summary, which wraps up all operations on the new
schema.
[06/23] Merge with [HDDS-5379|https://issues.apache.org/jira/browse/HDDS-5379]
Because we are intending to add a new CF to the Recon container DB, the
existing ContainerDBServiceProvider must be refactored to resolve conflict with
a new service provider sharing the same RDB as the current one directly
operates on the DB not just the CF/tables.
# Refactor ContainerDBServiceProvider into something not directly related to
the underlying RDB
# Use a new ReconDBStore instead of containerDbStore in
ContainerDBServiceProviderImpl
# Rename ContainerDBServiceProvider/ContainerDBServiceProviderImpl into sth
more appropriate
# Combined ReconRocksDB and ReconDBProvider into one class, which manages all
db-level operations.
was:
Table schema:
{code:java}
objectId --> class NSSummary
{code}
New class NSSummary should at least include these fields:
1. Total number of files *directly*[1] under this directory,
2. Total size of files directly under this directory,
3. File Size Buckets of files directly under this directory.
Note: [1] *directly* means not counting files under any subdirectories, for
now. We could further optimize the speed later by trading more writes
(propagating the statistics any layers deep upwards) with speed. This is for
reducing potential write-amplification for now. The stats could be extended
into arbitrary layers deep later.
[06/21] Renamed ReconContainerDBProvider to ReconDBProvider
[06/21] Expanded the scope of this issue by designing a service provider
interface for namespace summary, which wraps up all operations on the new
schema.
> Add a new column family and a service provider in Recon DB for Namespace
> Summaries
> ----------------------------------------------------------------------------------
>
> Key: HDDS-5332
> URL: https://issues.apache.org/jira/browse/HDDS-5332
> Project: Apache Ozone
> Issue Type: Sub-task
> Reporter: Siyao Meng
> Assignee: Yuan Gu
> Priority: Major
>
> Table schema:
> {code:java}
> objectId --> class NSSummary
> {code}
> New class NSSummary should at least include these fields:
> 1. Total number of files *directly*[1] under this directory,
> 2. Total size of files directly under this directory,
> 3. File Size Buckets of files directly under this directory.
> Note: [1] *directly* means not counting files under any subdirectories, for
> now. We could further optimize the speed later by trading more writes
> (propagating the statistics any layers deep upwards) with speed. This is for
> reducing potential write-amplification for now. The stats could be extended
> into arbitrary layers deep later.
> [06/21] Renamed ReconContainerDBProvider to ReconDBProvider
> [06/21] Expanded the scope of this issue by designing a service provider
> interface for namespace summary, which wraps up all operations on the new
> schema.
>
> [06/23] Merge with [HDDS-5379|https://issues.apache.org/jira/browse/HDDS-5379]
> Because we are intending to add a new CF to the Recon container DB, the
> existing ContainerDBServiceProvider must be refactored to resolve conflict
> with a new service provider sharing the same RDB as the current one directly
> operates on the DB not just the CF/tables.
> # Refactor ContainerDBServiceProvider into something not directly related to
> the underlying RDB
> # Use a new ReconDBStore instead of containerDbStore in
> ContainerDBServiceProviderImpl
> # Rename ContainerDBServiceProvider/ContainerDBServiceProviderImpl into sth
> more appropriate
> # Combined ReconRocksDB and ReconDBProvider into one class, which manages
> all db-level operations.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
