[
https://issues.apache.org/jira/browse/HDDS-13311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Swaminathan Balachandran updated HDDS-13311:
--------------------------------------------
Description:
DirectoryDeletingService should use rocksdb deleteRange instead of creating
individual tombstones which can cause seek time issue. But in the presence of
snapshots the deleteRange APi should stitch continuous key ranges together that
are reclaimable and not issue a blind deleteRange which could lead to incorrect
reclaimation of the entry and lead to unreference orphan blocks when the
snapshots are deleted.
DeleteRange APIs on FileTable, DirectoryTable, KeyTable can be used by
background garbage collection services and should never be used by user facing
APIs like keyDelete as that can cause issues in snapshot correctness.
E.g.
{code:java}
Dir1/Key1(Reclaimable) Dir1/Key2(Reclaimable) Dir1/key3(Not Reclaimable)
Dir1/Key4(Reclaimable) Dir1/Key5(Reclaimable){code}
Then DirectoryDeletingService should issue 2 delete range like
[Dir1/Key1..Dir1/Key2] (Both inclusive)
[Dir1/Key4..Dir1/Key5]
In terms of rocksdb deleteRange where the end key range is exclusive this
would be equivalent to
[Dir1/Key1..Dir1/Key3) and [Dir1/Key4..lexicographicalHigherString(Dir1/)]
was:
DirectoryDeletingService should use rocksdb deleteRange instead of creating
individual tombstones which can cause seek time issue. But in the presence of
snapshots the deleteRange APi should stitch continuous key ranges together that
are reclaimable and not issue a blind deleteRange which could lead to incorrect
reclaimation of the entry and lead to unreference orphan blocks when the
snapshots are deleted.
DeleteRange APIs on FileTable, DirectoryTable, KeyTable can be used by
background garbage collection services and should never be used by user facing
APIs like keyDelete as that can cause issues in snapshot correctness.
E.g.
{code:java}
Dir1/Key1(Reclaimable) Dir1/Key2(Reclaimable) Dir1/key3(Not Reclaimable)
Dir1/Key4(Reclaimable) Dir1/Key5(Reclaimable){code}
Then DirectoryDeletingService should issue 2 delete range like
[Dir1/Key1..Dir1/Key2] (Both inclusive)
[Dir1/Key4..Dir1/Key5]
In terms of rocksdb this would be equivalent to
[Dir1/Key1..Dir1/Key3) and [Dir1/Key4..lexicographicalHigherString(Dir1/)]
> Directory Deleting Service can use deleteRange to delete subDirectories and
> subFiles
> ------------------------------------------------------------------------------------
>
> Key: HDDS-13311
> URL: https://issues.apache.org/jira/browse/HDDS-13311
> Project: Apache Ozone
> Issue Type: Sub-task
> Reporter: Swaminathan Balachandran
> Assignee: Swaminathan Balachandran
> Priority: Major
>
> DirectoryDeletingService should use rocksdb deleteRange instead of creating
> individual tombstones which can cause seek time issue. But in the presence of
> snapshots the deleteRange APi should stitch continuous key ranges together
> that are reclaimable and not issue a blind deleteRange which could lead to
> incorrect reclaimation of the entry and lead to unreference orphan blocks
> when the snapshots are deleted.
> DeleteRange APIs on FileTable, DirectoryTable, KeyTable can be used by
> background garbage collection services and should never be used by user
> facing APIs like keyDelete as that can cause issues in snapshot correctness.
> E.g.
> {code:java}
> Dir1/Key1(Reclaimable) Dir1/Key2(Reclaimable) Dir1/key3(Not Reclaimable)
> Dir1/Key4(Reclaimable) Dir1/Key5(Reclaimable){code}
>
> Then DirectoryDeletingService should issue 2 delete range like
> [Dir1/Key1..Dir1/Key2] (Both inclusive)
> [Dir1/Key4..Dir1/Key5]
> In terms of rocksdb deleteRange where the end key range is exclusive this
> would be equivalent to
> [Dir1/Key1..Dir1/Key3) and [Dir1/Key4..lexicographicalHigherString(Dir1/)]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
