Chu Cheng Li created HDDS-17174:
-----------------------------------
Summary: KeyDeletingService should wait for the double buffer
flush after purging keys
Key: HDDS-17174
URL: https://issues.apache.org/jira/browse/HDDS-17174
Project: Apache Ozone
Issue Type: Improvement
Reporter: Chu Cheng Li
*Problem*
{{KeyDeletingService}} submits {{PurgeKeys}} requests through Ratis. A request
returns when OM applies it. At that time, the double buffer has not always
written the purge to RocksDB.
The purge request does not add cache entries for {{deletedTable}}. Thus, the
purged keys stay in {{deletedTable}} until the flush.
The service updates {{deletedKeyCount}} and its deletion metrics when the
requests return.
*Effect*
* A reader that waits for {{deletedKeyCount}} and then reads {{deletedTable}}
can still find purged keys.
* {{TestKeyDeletingService}} failed for this reason (HDDS-17077). The fix for
HDDS-17077 added {{awaitDoubleBufferFlush}} calls to the test.
* The next run checks {{isPreviousPurgeTransactionFlushed}}. If the last purge
is not flushed, the run skips the active object store.
*Fix*
* {{OzoneManagerDoubleBuffer}} records the index of the last transaction that
it adds.
* The new method {{OzoneManagerStateMachine.awaitAppliedTransactionsFlushed}}
waits until the double buffer flushes this transaction. If the flush is already
done, the method returns immediately.
* {{KeyDeletingService.processKeyDeletes}} calls this method after it submits
the purge requests. Thus, the counts that the service returns agree with the DB.
* {{TestKeyDeletingService}} does not wait for the flush after the service runs
anymore. This removes the two {{awaitDoubleBufferFlush}} calls that HDDS-17077
added.
* {{DirectoryDeletingService}} and {{SnapshotDeletingService}} do not change.
They submit requests while they keep snapshot DB handles open. The flush thread
can need a write lock on the same snapshot lock stripe. Thus, a wait for the
flush can deadlock until HDDS-16164 is fixed.
* {{isPreviousPurgeTransactionFlushed}} does not change. After a leader change,
the new leader can still have purges from the old leader that are not flushed.
Related: HDDS-17077.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
