Stephen O'Donnell created HDDS-11660:
----------------------------------------
Summary: Recon List Key API: Reduce object creation and buffering
memory
Key: HDDS-11660
URL: https://issues.apache.org/jira/browse/HDDS-11660
Project: Apache Ozone
Issue Type: Improvement
Components: Ozone Recon
Reporter: Stephen O'Donnell
Assignee: Stephen O'Donnell
The current ListKeysAPI implementation, pulls a batch of OMKeyInfo objects from
the File / Key table and buffers them in an array.
Then it converts each item in that array to a KeyEntityInfo object, which has
JSON annotations allowing the data to be serialized over HTTP.
This results in two copies of the objects buffered in memory at once.
Additionally, the OMKeyInfo object is a fairly heavyweight object, and creating
it from protobuf also deserialized the block information, which is not needed
for the ListKey purposes.
Further taking the protobuffer bytes from RocksDB creates a KeyInfo object
(deserialized proto) and then this object is copied into OmKeyInfo.
So for each key retrieved, we start with:
1. A deserialized proto object (KeyInfo)
2. A copy of the proto fields (OmKeyInfo)
3. A json annotated object (KeyEntityInfo).
By deserializing into the 3rd object directly, and having it wrap a keyInfo
object and delegate calls to it when appropriate, we avoid needing the
heavyweight OmKeyInfo object, and make the KeyEntityInfo object "slimmer". We
can also avoid the double buffering, creating only a single list of objects for
the batch.
This PR introduces a KeyEntityInfoProtoWrapper object, rather than changing the
existing KeyEntityInfo object. The reason, is that other APIs outside of
ListKeys use it. I am not ready to change those APIs now, however after this
change gets committed and we are happy with it, it should be possible to apply
similar changes to those other API and eventually remove KeyEntityInfo from the
system.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
