[
https://issues.apache.org/jira/browse/HDDS-12491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wei-Chiu Chuang resolved HDDS-12491.
------------------------------------
Resolution: Implemented
Resolving as implemented. Ordinary SST iteration and raw/tombstone SST
iteration now use reusable Buffer/CodecBuffer instances instead of fetching a
new key byte array for every entry. Buffer allocates direct memory; the raw
iterator JNI getKey/getValue methods accept ByteBuffer.
Evidence at d0949f291791c399f3e54409fcb92656a80597a4:
*
https://github.com/apache/ozone/blob/d0949f291791c399f3e54409fcb92656a80597a4/hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/utils/db/Buffer.java#L51
*
https://github.com/apache/ozone/blob/d0949f291791c399f3e54409fcb92656a80597a4/hadoop-hdds/rocksdb-checkpoint-differ/src/main/java/org/apache/hadoop/hdds/utils/db/ManagedSstFileIterator.java#L60
*
https://github.com/apache/ozone/blob/d0949f291791c399f3e54409fcb92656a80597a4/hadoop-hdds/rocks-native/src/main/java/org/apache/hadoop/hdds/utils/db/ManagedRawSSTFileIterator.java#L40
*
https://github.com/apache/ozone/blob/d0949f291791c399f3e54409fcb92656a80597a4/hadoop-hdds/rocksdb-checkpoint-differ/src/main/java/org/apache/hadoop/hdds/utils/db/SstFileSetReader.java#L155
> Snapshot diff should use direct byte buffers on SSTFileReader and
> RawSstFileReader while fetching keys
> ------------------------------------------------------------------------------------------------------
>
> Key: HDDS-12491
> URL: https://issues.apache.org/jira/browse/HDDS-12491
> Project: Apache Ozone
> Issue Type: Sub-task
> Reporter: Swaminathan Balachandran
> Assignee: Swaminathan Balachandran
> Priority: Major
>
> Currently SnapshotDiff creates a new byte array for each and every key read
> from sstFileReaderIterator & RawSstFileIterator which would cause a lot of
> unnecessary GC. We should use Direct ByteBuffer instead this wasn't done in
> the V1 since we wanted to first stabilise the native lib and then look for
> performance optimizations in this flow.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
