[
https://issues.apache.org/jira/browse/HDDS-12501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Swaminathan Balachandran updated HDDS-12501:
--------------------------------------------
Description:
Currently backup sst directories just stores the hard links to the input sst
files on compaction. We should eventually rewrite the sst file(Atomic rewrite)
and just store the key in the sst file and get rid of OMKeyInfo. This will
significantly reduce the size of the directory significantly reducing the
storage footprint of the sst backup directory. Currently an average of
OMKeyInfo is in the order of few KBs or even more and the average key would be
in the order of 50 bytes at most.
While rewriting the sst file it should be kept in mind we read the tombstones
in the sst file using the rawSstFileReader and write it back with sstFileWriter
was:Currently backup sst directories just stores the hard links to the input
sst files on compaction. We should eventually rewrite the sst file(Atomic
rewrite) and just store the key in the sst file and get rid of OMKeyInfo. This
will significantly reduce the size of the directory significantly reducing the
storage footprint of the sst backup directory. Currently an average of
OMKeyInfo is in the order of few KBs or even more and the average key would be
in the order of 50 bytes at most.
> Background thread in RocksDBCheckpointDiffer should convert backup sst files
> to store only key and get rid of OmKeyInfo
> -----------------------------------------------------------------------------------------------------------------------
>
> Key: HDDS-12501
> URL: https://issues.apache.org/jira/browse/HDDS-12501
> Project: Apache Ozone
> Issue Type: Sub-task
> Reporter: Swaminathan Balachandran
> Priority: Major
>
> Currently backup sst directories just stores the hard links to the input sst
> files on compaction. We should eventually rewrite the sst file(Atomic
> rewrite) and just store the key in the sst file and get rid of OMKeyInfo.
> This will significantly reduce the size of the directory significantly
> reducing the storage footprint of the sst backup directory. Currently an
> average of OMKeyInfo is in the order of few KBs or even more and the average
> key would be in the order of 50 bytes at most.
> While rewriting the sst file it should be kept in mind we read the tombstones
> in the sst file using the rawSstFileReader and write it back with
> sstFileWriter
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
