Chu Cheng Li created HDDS-17465:
-----------------------------------
Summary: Recon keeps the old container mappings of an overwritten
key
Key: HDDS-17465
URL: https://issues.apache.org/jira/browse/HDDS-17465
Project: Apache Ozone
Issue Type: Bug
Components: Ozone Recon
Reporter: Chu Cheng Li
Assignee: Chu Cheng Li
h3. Problem
* For an UPDATE event (a key overwrite), {{ContainerKeyMapperHelper#process}}
deletes the old container mappings of {{getOldValue().getKeyName()}}. This is
the key name or the file name.
* {{containerKeyTable}} and {{keyContainerTable}} use the OM DB key, for
example {{/vol/bucket/key}} or {{/volumeId/bucketId/parentId/file}}. Thus, the
delete finds nothing.
* The old mappings stay, and the key counts of the old containers stay too
high. A replay of 100K OM transactions on a 1.25M-key OM DB left 24,959 stale
container-key rows. A replay of 1M transactions on a 100M-key OM DB left
230,478.
h3. Changes
* {{ContainerKeyMapperHelper}}:
** An UPDATE deletes the old mappings by the OM DB key of the event.
** The put step adds a mapping again when an earlier event in the same batch
deleted it. The Recon DB keeps this mapping until the batch is written. Without
this step, an update that keeps a block in the same container loses its
mapping. An update of metadata only, for example of ACLs, has the same problem.
** The delete step skips the mappings that an earlier event in the same batch
deleted. Thus, when one batch overwrites a key two times, the count of the old
container goes down one time.
** The deleted mappings of a batch are in a {{HashSet}}, not an {{ArrayList}},
because the put and delete steps look them up.
h3. Tests
* An overwrite that moves a key from containers 1 and 2 to containers 2 and 3
removes the key from container 1 only. The key counts of the three containers
are correct.
* Two overwrites of a key in one batch, from container 1 to 2 and then to 3,
decrease the count of container 1 one time.
* Both tests run for an FSO file and an OBS key.
Found in a comparison of the incremental results of Recon with a rebuild of the
final OM DB.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
