[
https://issues.apache.org/jira/browse/HDDS-17173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chu Cheng Li reassigned HDDS-17173:
-----------------------------------
Assignee: Chu Cheng Li
> OM table iterators should return applied entries that are not flushed yet
> -------------------------------------------------------------------------
>
> Key: HDDS-17173
> URL: https://issues.apache.org/jira/browse/HDDS-17173
> Project: Apache Ozone
> Issue Type: Improvement
> Reporter: Chu Cheng Li
> Assignee: Chu Cheng Li
> Priority: Major
>
> *Problem*
> OM applies a write request in two steps. First, {{validateAndUpdateCache}}
> updates the table cache. Later, the double buffer writes the change to
> RocksDB.
> {{Table.get}} reads the table cache first. Thus, it returns the applied state.
> {{Table.iterator}} reads RocksDB only. Thus, it does not return an entry that
> is applied but not flushed. It also returns an entry that the cache marks as
> deleted. {{keyIterator}}, {{valueIterator}} and
> {{OmMetadataManagerImpl.countRowsInTable}} use {{Table.iterator}}. Thus, they
> have the same problem.
> *Effect*
> * Each caller that iterates a table must merge the cache by hand. OM has
> about 20 of these merges, for example in {{ListIterator}},
> {{KeyManagerImpl.listStatus}} and {{OMFileRequest.hasChildren}}. A merge that
> is missing or wrong is a bug, for example HDDS-14600 and HDDS-9233.
> * Some callers do not merge the cache. For example,
> {{OmMetadataManagerImpl.listOpenFiles}} does not show a new open key until
> the flush.
> * A test that reads a table right after a request can see old data. Thus,
> tests add {{awaitDoubleBufferFlush}} calls, for example in HDDS-16523,
> HDDS-13865 and HDDS-13880.
> *Fix*
> * {{TypedTable.iterator}} also returns the cache entries that RocksDB does
> not contain yet. A cache value replaces the DB value of the same key. A
> deleted cache entry hides the DB entry. The merge uses the RocksDB key order
> (unsigned bytes), not {{String.compareTo}}.
> * The new class {{FlushedCacheEpoch}} records the highest cache epoch in each
> committed batch. When a DB iterator opens, it gets the epoch that it
> contains. The iterator merges only the cache entries with a higher epoch. If
> the open overlaps a commit, the open runs again.
> * The epoch check is necessary for two reasons. First, a cache entry stays in
> the cache for a short time after the flush. Second, snapshot create deletes a
> range of {{deletedTable}} at flush time, without a cache entry. Without the
> check, an iterator can return keys that moved to the snapshot.
> * Only a store built with {{DBStoreBuilder.setIteratorsReadTableCache(true)}}
> merges the cache. The OM DB uses this setting. SCM, Recon, datanodes and OM
> snapshot DBs do not change. Full-cache tables (volume and bucket) do not
> change.
> * The double buffer commits each batch with {{commitBatchOperation(batch,
> lastIndex)}}. It builds the batch inside {{FlushedCacheEpoch.readDbOnly}}.
> Thus, the responses still read the DB as of the previous batch. They do not
> read the cache entries of later transactions.
> * The checkpoint servlet reads the snapshot directories from the DB only. A
> snapshot directory exists only after the flush.
> * {{KeyLifecycleService.getSubDirectory}} does not scan the cache separately
> anymore. Otherwise, it counts each cached sub-directory two times.
> * {{TestHSync}} does not call {{awaitDoubleBufferFlush}} before it reads
> {{openFileTable}} anymore (HDDS-16523).
> * Requests do not add cache entries for {{deletedTable}}, {{deletedDirTable}}
> and {{snapshotRenamedTable}}. Thus, this change does not affect these tables.
> Related: HDDS-17077.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
