[
https://issues.apache.org/jira/browse/HDDS-16943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HDDS-16943:
----------------------------------
Labels: pull-request-available (was: )
> Avoid the extra ListStatus RPC at the end of every file system listing
> ----------------------------------------------------------------------
>
> Key: HDDS-16943
> URL: https://issues.apache.org/jira/browse/HDDS-16943
> Project: Apache Ozone
> Issue Type: Improvement
> Components: Ozone Manager
> Reporter: Andrey Yarovoy
> Assignee: Andrey Yarovoy
> Priority: Major
> Labels: pull-request-available
>
> h2. Problem
> The file system clients keep listing until OM returns a page that adds no new
> entry:
> {code:java}
> // BasicRootedOzoneFileSystem.listStatusAdapter (same loop in
> BasicOzoneFileSystem)
> do {
> tmpStatusList = adapter.listStatus(pathToKey(f), false, startPath,
> numEntries, ...);
> // add entries, dropping a repeated first entry
> ...
> } while (entriesAdded > 0);
> {code}
> So every \{{FileSystem.listStatus}} costs at least two OM listing RPCs, even
> when the whole directory fits in the
> first page. The second call returns only the repeated start key (OM includes
> \{{startKey}} in the result) and adds
> nothing. The \{{listStatusIterator}} path (\{{hasNextNoFilter}}) has the same
> rule: it fetches another page whenever
> the current one was not empty.
> In a benchmark of 2000 \{{ofs}} \{{listStatus}} calls on directories of 10
> entries (\{{TestOfsListStatusRpcBenchmark}},
> HDDS-16760), every call made exactly two ListStatusLight RPCs. One of them is
> not needed.
> h2. Why the client cannot stop on a short page
> Before HDDS-11696 the loop stopped when a page was shorter than
> \{{numEntries}}. HDDS-11696 changed it to stop on an
> empty page, because OM caps the page size on its own:
> * \{{OzoneManagerRequestHandler.limitListSize}}: \{{min(numEntries,
> ozone.om.server.list.max.size)}} (default *1000*);
> * \{{OmMetadataReader.listStatus}} / \{{listStatusLight}}: \{{min(n,
> ozone.fs.listing.page.size.max)}} (default 1024),
> raised to at least 2 by \{{OzoneConfigUtil.limitValue}}.
> The client asks for \{{ozone.fs.listing.page.size}} (default *1024*), so with
> default settings a full page of 1000
> entries is shorter than the request. The client cannot know OM's caps, which
> are server configuration, so a
> client-only "stop on a short page" rule would silently cut off any directory
> with more than 1000 entries. This
> must not be reintroduced.
> h2. Proposal
> Let OM say whether the listing may continue, as \{{ListKeysResponse}} /
> \{{ListKeysLightResponse}} already do with
> {\{isTruncated}}.
> # *Proto.* Add \{{optional bool isTruncated}} to \{{ListStatusResponse}} and
> \{{ListStatusLightResponse}}.
> # *OM.* Set it to \{{returnedCount >= effectiveLimit}}, where
> \{{effectiveLimit}} is the page size OM actually used
> after both caps above. Compare with that value, not with the requested
> \{{numEntries}}.
> Both listing paths return \{{min(effectiveLimit, remaining live entries)}}:
> ** FSO, \{{OzoneListStatusHelper.listStatusFSO}}: a single loop {{while
> (map.size() < numEntries &&
> heapIterator.hasNext())}}; cache tombstones and DB keys shadowed by the
> cache never reach the heap, so they do
> not use up the limit.
> ** LEGACY/OBS, \{{KeyManagerImpl.listStatus}}: entries are counted only when
> a live status is added, and the final
> loop stops at \{{numEntries}}.
> So \{{isTruncated = false}} always means the listing is complete. When
> exactly \{{effectiveLimit}} entries remain,
> the flag is \{{true}} and the client makes one more call, as it does today.
> A look-ahead
> (\{{heapIterator.hasNext()}} on FSO, remaining map/iterator entries on
> LEGACY) could avoid that last call, but it is
> not needed for correctness.
> # *Client.* Stop after a page whose response has \{{isTruncated}} set to
> \{{false}}. If the field is absent (an older
> OM), keep the current loop. Field presence distinguishes the two, so no
> \{{OzoneManagerVersion}} gate is needed.
> # *Plumbing.* The listing calls return a plain \{{List}} today through
> \{{OzoneManagerProtocol}},
> \{{ClientProtocol}} / \{{RpcClient}}, \{{OzoneClientAdapter}} and the two
> file systems. The flag needs a way up,
> for example a small result type holding the entries and the flag, used by
> the file system listing loops and by
> \{{hasNextNoFilter}}. Existing \{{List}}-returning methods can stay for
> other callers.
> Both \{{ofs}} (\{{BasicRootedOzoneFileSystem}}) and \{{o3fs}}
> (\{{BasicOzoneFileSystem}}) benefit.
> h2. Expected effect
> A listing that fits in one page goes from two OM listing RPCs to one. On top
> of HDDS-16760, an \{{ofs}}
> {\{listStatus}} of such a directory goes from 2 OM RPCs to 1; before
> HDDS-16760 it was 4.
> h2. Compatibility
> * One new optional field in two response messages; no new request field,
> result code or configuration.
> * New client, old OM: the field is absent, so the client keeps the current
> loop.
> * Old client, new OM: the field is ignored; behavior unchanged.
> * Listing results are unchanged; only the number of calls changes.
> h2. Testing
> * OM unit tests for both \{{ListStatus}} and \{{ListStatusLight}}, FSO and
> LEGACY: \{{isTruncated}} is \{{false}} when
> fewer than the effective limit entries are returned and \{{true}} on a full
> page, with the effective limit set by
> each cap (\{{ozone.om.server.list.max.size}} below and above
> \{{ozone.fs.listing.page.size.max}}, and a request of
> size 1, which OM raises to 2).
> * A full page whose only remaining entries are cache tombstones still ends
> the listing on the next call.
> * Client unit tests: the loop stops after one call when \{{isTruncated}} is
> \{{false}}, and keeps the current loop
> when the field is absent.
> * Integration: a directory larger than \{{ozone.om.server.list.max.size}}
> with the client page size above it, listed
> in full through \{{ofs}} and \{{o3fs}}, by \{{listStatus}} and
> \{{listStatusIterator}}. This is the regression guard
> for HDDS-11696.
> * \{{TestOfsListStatusRpcBenchmark}}: ListStatus RPCs per call drop from 2 to
> 1.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
