sodonnel commented on PR #5364:
URL: https://github.com/apache/ozone/pull/5364#issuecomment-1735950667
So we start with a byteString in the protobuf. Then we convert that into a
ByteBuffer, back to a byteString and back to a ByteBuffer and then in this PR a
final copy from ByteBuffer to ByteBuffer!
Were you seeing excessive memory usage on the DN side when writing chunks
and with chunk.data.validation.check=true (the default is false, which is
frankly crazy and will result in dataloss some day).
Ok - so it seems that `ByteString.asReadOnlyByteBufferList()` simply wraps
the internal ByteBuffer inside the ByteString.
Then `data.toByteString(byteBufferToByteString)` uses this function:
```
byteBufferToByteString =
ByteStringConversion.createByteBufferConversion(isUnsafeByteBufferConversionEnabled);
```
`isUnsafeByteBufferConversionEnabled` defaults to true, so this again
creates a ByteString which wraps the ByteBuffer and doesn't copy. So it seems
like all these copies are not copies at all.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
