snvijaya commented on code in PR #6069:
URL: https://github.com/apache/hadoop/pull/6069#discussion_r1332562001
##########
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClient.java:
##########
@@ -1412,6 +1432,91 @@ private void appendIfNotEmpty(StringBuilder sb, String
regEx,
}
}
+ /**
+ * Add MD5 hash as request header to the append request
+ * @param requestHeaders
+ * @param reqParams
+ * @param buffer
+ */
+ private void addCheckSumHeaderForWrite(List<AbfsHttpHeader> requestHeaders,
+ final AppendRequestParameters reqParams, final byte[] buffer)
+ throws AbfsRestOperationException {
+ try {
+ MessageDigest md5Digest = MessageDigest.getInstance(MD5);
+ byte[] dataToBeWritten = new byte[reqParams.getLength()];
+ System.arraycopy(buffer, reqParams.getoffset(), dataToBeWritten, 0,
reqParams.getLength());
Review Comment:
Arraycopies are costly. Given caller has the info for buffer offset and
length, it might be possible to identify cases where whole buffer is relevant
or if its a subset. If possible executing array copy only when subset is
relevant and not the whole buffer.
Same for the case of read checksum validation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
