swamirishi commented on code in PR #9549:
URL: https://github.com/apache/ozone/pull/9549#discussion_r2648659239
##########
hadoop-hdds/managed-rocksdb/src/main/java/org/apache/hadoop/hdds/utils/db/managed/ManagedDirectSlice.java:
##########
@@ -17,98 +17,34 @@
package org.apache.hadoop.hdds.utils.db.managed;
-import static org.apache.hadoop.hdds.utils.db.managed.ManagedRocksDB.NOT_FOUND;
-
-import com.google.common.annotations.VisibleForTesting;
import java.nio.ByteBuffer;
-import org.apache.hadoop.hdds.utils.db.RocksDatabaseException;
-import org.apache.ratis.util.function.CheckedConsumer;
-import org.apache.ratis.util.function.CheckedFunction;
import org.rocksdb.DirectSlice;
-import org.rocksdb.RocksDBException;
/**
- * ManagedDirectSlice is a managed wrapper around the DirectSlice object. It
ensures
- * proper handling of native resources associated with DirectSlice, utilizing
- * the ManagedObject infrastructure to prevent resource leaks. It works in
tandem
- * with a ByteBuffer, which acts as the data source for the managed slice.
+ * ManagedDirectSlice is a class that extends the {@link DirectSlice} class
and provides additional
+ * management for slices of direct {@link ByteBuffer} memory. This class
initializes the slice with
+ * the given ByteBuffer and sets its prefix and length properties based on the
buffer's position
+ * and remaining capacity.
+ *
+ * The class is designed to handle specific memory slicing operations while
ensuring that the
+ * provided ByteBuffer’s constraints are respected. ManagedDirectSlice
leverages its parent
+ * {@link DirectSlice} functionalities to deliver optimized direct buffer
handling.
+ *
+ * Constructor:
+ * - Initializes the ManagedDirectSlice instance with a provided ByteBuffer.
+ * - Sets the slice length to the buffer's remaining capacity.
+ * - Removes the prefix based on the buffer's position.
*
- * This class overrides certain operations to tightly control the lifecycle and
- * behavior of the DirectSlice it manages. It specifically caters to use cases
- * where the slice is used in RocksDB operations, providing methods for safely
- * interacting with the slice for put-like operations.
+ * NOTE: This class should be only with ByteBuffer whose position and limit is
going be immutable in the lifetime of
+ * this ManagedDirectSlice instance. This means that the ByteBuffer's
position and limit should not be modified
+ * externally while the ManagedDirectSlice is in use. The value in the byte
buffer should be only accessed via the
+ * instance.
*/
-public class ManagedDirectSlice extends ManagedObject<DirectSlice> {
-
- private final ByteBuffer data;
+public class ManagedDirectSlice extends DirectSlice {
Review Comment:
This is the benchmark code I wrote:
`/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.hadoop.hdds.utils.db;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import org.apache.commons.lang3.RandomUtils;
import org.apache.hadoop.hdds.utils.db.managed.ManagedDirectSlice;
import org.apache.hadoop.hdds.utils.db.managed.ManagedRocksObjectUtils;
import org.apache.hadoop.hdds.utils.db.managed.ManagedSlice;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.ValueSource;
public class TestRDBBatchOperation {
@BeforeAll
public static void loadLibrary() {
ManagedRocksObjectUtils.loadRocksDBLibrary();
}
@ParameterizedTest
@ValueSource(ints = {1, 10, 20, 40, 100, 1000})
public void testManagedDirectSliceCreate(int numberOfBytesWritten) throws
CodecException {
int numberOfKeys = 1000000;
byte[] randomBytes =
RandomUtils.secure().nextBytes(numberOfBytesWritten);
List<CodecBuffer> buffers = new ArrayList<>(numberOfKeys);
for (int i = 0; i < numberOfKeys; i++) {
buffers.add(CodecBufferCodec.get(true).fromPersistedFormat(randomBytes));
}
List<ManagedDirectSlice> slices = new ArrayList<>(numberOfKeys);
long startTime = System.currentTimeMillis();
for (CodecBuffer buffer : buffers) {
slices.add(new ManagedDirectSlice(buffer.asReadOnlyByteBuffer()));
}
long totalTime = System.currentTimeMillis() - startTime;
System.out.println("Time taken to create ManagedDirectSlice: " +
totalTime + " ms with key length: " + numberOfBytesWritten);
slices.forEach(ManagedDirectSlice::close);
buffers.forEach(CodecBuffer::close);
}
@ParameterizedTest
@ValueSource(ints = {1, 10, 20, 40, 100, 1000})
public void testManagedSliceCreate(int numberOfBytesWritten) throws
CodecException {
int numberOfKeys = 1000000;
byte[] randomBytes =
RandomUtils.secure().nextBytes(numberOfBytesWritten);
List<byte[]> buffers = new ArrayList<>(numberOfKeys);
for (int i = 0; i < numberOfKeys; i++) {
buffers.add(Arrays.copyOf(randomBytes, randomBytes.length));
}
List<ManagedSlice> slices = new ArrayList<>(numberOfKeys);
long startTime = System.currentTimeMillis();
for (byte[] buffer : buffers) {
slices.add(new ManagedSlice(buffer));
}
long totalTime = System.currentTimeMillis() - startTime;
System.out.println("Time taken to create Slice: " + totalTime + " ms
with key length: " + numberOfBytesWritten);
slices.forEach(ManagedSlice::close);
}
}
`
Result for creating 1 million Slices and DirectSlices:
```
Time taken to create ManagedDirectSlice: 262 ms with key length: 1
Time taken to create ManagedDirectSlice: 168 ms with key length: 10
Time taken to create ManagedDirectSlice: 810 ms with key length: 20
Time taken to create ManagedDirectSlice: 292 ms with key length: 40
Time taken to create ManagedDirectSlice: 160 ms with key length: 100
Time taken to create ManagedDirectSlice: 339 ms with key length: 1000
Time taken to create Slice: 675 ms with key length: 1
Time taken to create Slice: 509 ms with key length: 10
Time taken to create Slice: 527 ms with key length: 20
Time taken to create Slice: 531 ms with key length: 40
Time taken to create Slice: 531 ms with key length: 100
Time taken to create Slice: 1663 ms with key length: 1000
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
