[
https://issues.apache.org/jira/browse/HDDS-9290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Siyao Meng updated HDDS-9290:
-----------------------------
Description:
To quote [~sodonnell]'s comment:
bq. You could also change the OMKeyInfo to wrap the proto object rather than
duplicating all the data into the object and throwing the proto object away.
h2. Background
Right now, when converting from protobuf obj to DAO, e.g.
[{{OmKeyInfo#getFromProtobuf}}|https://github.com/apache/ozone/blob/69f700bbcf5eb90afcf26dd59065e1f73ea29c1a/hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/OmKeyInfo.java#L699]
is copying all protobuf fields one by one into the {{OmKeyInfo}}'s internal
correspondingly defined fields (using {{Builder}}).
h2. Proposal
We could change the DAO ({{OmKeyInfo}} and others) so that the whole proto
object ({{OzoneManagerProtocolProtos.KeyInfo}} directly lives inside the DAO.
So we won't have to do the unnecessary step of copying the fields one by one in
the first place. And because the change is in the DAO internally, there will be
*minimal impact* to the callers (unless the caller is modifying the referenced
proto object, which will need to be adjusted in such cases).
This could potentially reduce the allocate/GC rate by a lot. Might slightly
improvement read/write performance as well.
We could start by making the change in hotspot DAOs like OmKeyInfo then
gradually roll it out to other DAOs.
cc [~swagle] [~szetszwo] [~adoroszlai] [~ritesh] [~weichiu] [~erose]
was:
To quote [~sodonnell]'s comment:
bq. You could also change the OMKeyInfo to wrap the proto object rather than
duplicating all the data into the object and throwing the proto object away.
h2. Background
Right now, when converting from protobuf obj to DAO, e.g.
[{{OmKeyInfo#getFromProtobuf}}|https://github.com/apache/ozone/blob/69f700bbcf5eb90afcf26dd59065e1f73ea29c1a/hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/OmKeyInfo.java#L699]
is copying all protobuf fields one by one into the {{OmKeyInfo}}'s internal
correspondingly defined fields (using {{Builder}}).
h2. Proposal
We could change the DAO ({{OmKeyInfo}} and others) so that the whole proto
object ({{OzoneManagerProtocolProtos.KeyInfo}} directly lives inside the DAO.
So we won't have to do the unnecessary step of copying the fields one by one in
the first place. And because the change is in the DAO internally, there will be
*minimal impact* to the callers (unless the caller is modifying the referenced
proto object, which will need to be adjusted in such cases).
This could potentially reduce the allocate/GC rate by a lot.
We could start by making the change in hotspot DAOs like OmKeyInfo then
gradually roll it out to other DAOs.
cc [~swagle] [~szetszwo] [~adoroszlai] [~ritesh] [~weichiu] [~erose]
> Wrap Protobuf object directly inside of DAOs to avoid copying whenever
> possible
> -------------------------------------------------------------------------------
>
> Key: HDDS-9290
> URL: https://issues.apache.org/jira/browse/HDDS-9290
> Project: Apache Ozone
> Issue Type: Task
> Reporter: Siyao Meng
> Priority: Major
>
> To quote [~sodonnell]'s comment:
> bq. You could also change the OMKeyInfo to wrap the proto object rather than
> duplicating all the data into the object and throwing the proto object away.
> h2. Background
> Right now, when converting from protobuf obj to DAO, e.g.
> [{{OmKeyInfo#getFromProtobuf}}|https://github.com/apache/ozone/blob/69f700bbcf5eb90afcf26dd59065e1f73ea29c1a/hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/OmKeyInfo.java#L699]
> is copying all protobuf fields one by one into the {{OmKeyInfo}}'s internal
> correspondingly defined fields (using {{Builder}}).
> h2. Proposal
> We could change the DAO ({{OmKeyInfo}} and others) so that the whole proto
> object ({{OzoneManagerProtocolProtos.KeyInfo}} directly lives inside the DAO.
> So we won't have to do the unnecessary step of copying the fields one by one
> in the first place. And because the change is in the DAO internally, there
> will be *minimal impact* to the callers (unless the caller is modifying the
> referenced proto object, which will need to be adjusted in such cases).
> This could potentially reduce the allocate/GC rate by a lot. Might slightly
> improvement read/write performance as well.
> We could start by making the change in hotspot DAOs like OmKeyInfo then
> gradually roll it out to other DAOs.
> cc [~swagle] [~szetszwo] [~adoroszlai] [~ritesh] [~weichiu] [~erose]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
