[
https://issues.apache.org/jira/browse/HDDS-17465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HDDS-17465:
----------------------------------
Labels: pull-request-available (was: )
> Recon keeps the old container mappings of an overwritten key
> ------------------------------------------------------------
>
> Key: HDDS-17465
> URL: https://issues.apache.org/jira/browse/HDDS-17465
> Project: Apache Ozone
> Issue Type: Bug
> Components: Ozone Recon
> Reporter: Chu Cheng Li
> Assignee: Chu Cheng Li
> Priority: Major
> Labels: pull-request-available
>
> h3. Problem
> * For an UPDATE event (a key overwrite), {{ContainerKeyMapperHelper#process}}
> deletes the old container mappings of {{getOldValue().getKeyName()}}. This is
> the key name or the file name.
> * {{containerKeyTable}} and {{keyContainerTable}} use the OM DB key, for
> example {{/vol/bucket/key}} or {{/volumeId/bucketId/parentId/file}}. Thus,
> the delete finds nothing.
> * The old mappings stay, and the key counts of the old containers stay too
> high. A replay of 100K OM transactions on a 1.25M-key OM DB left 24,959 stale
> container-key rows. A replay of 1M transactions on a 100M-key OM DB left
> 230,478.
> h3. Changes
> * {{ContainerKeyMapperHelper}}:
> ** An UPDATE deletes the old mappings by the OM DB key of the event.
> ** The put step adds a mapping again when an earlier event in the same batch
> deleted it. The Recon DB keeps this mapping until the batch is written.
> Without this step, an update that keeps a block in the same container loses
> its mapping. An update of metadata only, for example of ACLs, has the same
> problem.
> ** The delete step skips the mappings that an earlier event in the same batch
> deleted. Thus, when one batch overwrites a key two times, the count of the
> old container goes down one time.
> ** The deleted mappings of a batch are in a {{HashSet}}, not an
> {{ArrayList}}, because the put and delete steps look them up.
> h3. Tests
> * An overwrite that moves a key from containers 1 and 2 to containers 2 and 3
> removes the key from container 1 only. The key counts of the three containers
> are correct.
> * Two overwrites of a key in one batch, from container 1 to 2 and then to 3,
> decrease the count of container 1 one time.
> * Both tests run for an FSO file and an OBS key.
> Found in a comparison of the incremental results of Recon with a rebuild of
> the final OM DB.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
