Stephen O'Donnell created HDDS-6315:
---------------------------------------
Summary: ListContainer API should use the sorted treeMaps more
effectively
Key: HDDS-6315
URL: https://issues.apache.org/jira/browse/HDDS-6315
Project: Apache Ozone
Issue Type: Improvement
Reporter: Stephen O'Donnell
As also reported in HDDS-6314, SCMClientProtocolServer.listContainer, the code
does not seem to use the fact that the "container lists by state" are already
sorted and in TreeMaps, so they could perform the paging much more efficiently.
I fear we are potentially forming some very large lists twice per call, only to
return a small slice of the list each time.
For example, the code looks like:
{code:java}
return scm.getContainerManager().getContainers(state).stream()
.filter(info -> info.containerID().getId() >= startContainerID)
.sorted().limit(count).collect(Collectors.toList()); {code}
The `sorted()` call must filter and process the entire list before the limit
can be applied, but as the list is stored in a treeMap, we should be able to
avoid the sort.
Better, a TreeMap allows you to obtain a sublist after a certain element using
some combination of "floorKey" and "subMap", so we may be able to perform this
paging much more efficiently.
Note that the `ContainerManagerImpl.getContainers(...)` API used by this class
is vulnerable to ConcurrentModificationException, and hence that needs resolved
either before or as part of addressing this change.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
