bshashikant commented on a change in pull request #2883:
URL: https://github.com/apache/ozone/pull/2883#discussion_r762799825
##########
File path:
hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/BlockDataStreamOutput.java
##########
@@ -260,16 +270,66 @@ public void write(ByteBuffer b, int off, int len) throws
IOException {
int curLen = len;
// set limit on the number of bytes that a ByteBuffer(StreamBuffer) can
hold
int maxBufferLen = config.getDataStreamMaxBufferSize();
- while (curLen > 0) {
- int writeLen = Math.min(curLen, maxBufferLen);
- final StreamBuffer buf = new StreamBuffer(b, off, writeLen);
- off += writeLen;
- bufferList.add(buf);
- writeChunkToContainer(buf.duplicate());
- curLen -= writeLen;
- writtenDataLength += writeLen;
- doFlushIfNeeded();
+ int minPacketSize = config.getDataStreamMinPacketSize();
+ int totalSmallBufferLen = getTotalSmallBufferLen(smallBuffers);
+ // add the buffer to smallBufferQueue if the size is lesser than
+ // minPacketSize
+ if (len + totalSmallBufferLen < minPacketSize) {
+ smallBuffers.add(b);
+ } else {
+ ByteBuffer prev = getPreviousBufferIfAny();
+ if (!BufferUtils.isEmpty(prev)) {
+ curLen += prev.position();
+ }
+ while (curLen > 0) {
+ int writeLen = Math.min(curLen, maxBufferLen);
+ final StreamBuffer buf;
+ if (!BufferUtils.isEmpty(prev)) {
+ prev.rewind();
+ ByteBuffer combinedBuffer =
+ ByteBuffer.allocate(prev.limit() + b.limit());
Review comment:
It's better to allocate buffers of size equal to packet size and keep on
using the same buffer until the packet is full or stream is closed/flushed.
This will help reducing buffer allocations call.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
