adoroszlai commented on a change in pull request #551: HDDS-2717. Handle chunk
increments in datanode
URL: https://github.com/apache/hadoop-ozone/pull/551#discussion_r386874032
##########
File path:
hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/BlockOutputStream.java
##########
@@ -274,13 +271,16 @@ public void writeOnRetry(long len) throws IOException {
if (len == 0) {
return;
}
- int count = 0;
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("Retrying write length {} for blockID {}", len, blockID);
+ }
Preconditions.checkArgument(len <= streamBufferMaxSize);
+ int count = 0;
while (len > 0) {
- long writeLen;
- writeLen = Math.min(chunkSize, len);
- if (writeLen == chunkSize) {
- writeChunk(bufferPool.getBuffer(count));
+ ChunkBuffer buffer = bufferPool.getBuffer(count);
+ long writeLen = Math.min(buffer.position(), len);
+ if (!buffer.hasRemaining()) {
Review comment:
I think so. By sending each buffer only once, we can simply send the
complete buffer. If we sent it incrementally, we would need to keep track of
the last write end position separately.
https://github.com/apache/hadoop-ozone/blob/9eb8e0ef28db64e573feec5bb652228e4add522d/hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/BlockOutputStream.java#L452
This is not new behavior, previously it checked `writeLen == chunkSize`,
which resulted in the same:
https://github.com/apache/hadoop-ozone/blob/9eb8e0ef28db64e573feec5bb652228e4add522d/hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/BlockOutputStream.java#L282-L283
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
