[
https://issues.apache.org/jira/browse/HDDS-7951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hemant Kumar updated HDDS-7951:
-------------------------------
Description:
Problem: For snapshot diff report persistence, we use RocksDB. Since RocksDB
doesn't expose any way to get *ColumnFamilyHandle* for existing table which is
needed to support pagination and we decided to have a global table
*snap-diff-report-table* to store diff report where key will be prefixed with
*ReportId* or *JobId*. Over the time this table will grow and will needed to
clean.
Task: Expectation from this is to remove entries from *snap-diff-report-table*
for a job once persistence time is over (like 7 or 30 days which is
configurable) or job failed due OM crash or any other reason.
Approach: One approach could be following.
We have another table *snap-diff-job-table* which can be used to determine
which job and thier report can be removed. It would be based on time instance
(creation time) and the job status (FAILED). Iterator over this table and move
entries from table: *snap-diff-job-table* to some new table (may be something
*delete-snap-diff-report-table*). Later deleted all the reports related to
entries in table *delete-snap-diff-report-table*.
> Clean diff report table
> -----------------------
>
> Key: HDDS-7951
> URL: https://issues.apache.org/jira/browse/HDDS-7951
> Project: Apache Ozone
> Issue Type: Sub-task
> Reporter: Hemant Kumar
> Priority: Major
>
> Problem: For snapshot diff report persistence, we use RocksDB. Since RocksDB
> doesn't expose any way to get *ColumnFamilyHandle* for existing table which
> is needed to support pagination and we decided to have a global table
> *snap-diff-report-table* to store diff report where key will be prefixed with
> *ReportId* or *JobId*. Over the time this table will grow and will needed to
> clean.
> Task: Expectation from this is to remove entries from
> *snap-diff-report-table* for a job once persistence time is over (like 7 or
> 30 days which is configurable) or job failed due OM crash or any other reason.
> Approach: One approach could be following.
> We have another table *snap-diff-job-table* which can be used to determine
> which job and thier report can be removed. It would be based on time instance
> (creation time) and the job status (FAILED). Iterator over this table and
> move entries from table: *snap-diff-job-table* to some new table (may be
> something *delete-snap-diff-report-table*). Later deleted all the reports
> related to entries in table *delete-snap-diff-report-table*.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
