peterxcli commented on code in PR #11427:
URL: https://github.com/apache/ozone/pull/11427#discussion_r4214531998
##########
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/impl/KeyValueStreamDataChannel.java:
##########
@@ -141,30 +142,45 @@ protected void cleanupInternal() throws IOException {
* Note that the PutBlock proto at the end is ignored; see HDDS-12007.
*/
private void writeBuffers() throws IOException {
- final ReferenceCountedObject<ByteBuf> ref = buffers.pollAll();
- final ByteBuf buf = ref.retain();
- try {
- setEndIndex(buf);
- // write the remaining data
- writeFully(buf.nioBuffer(), super::writeFileChannel);
- } finally {
- ref.release();
+ pollAllAndWrite(buffers, super::writeFileChannel, b -> {
+ setEndIndex(b);
+ return null;
+ });
+ }
+
+ /**
+ * Write all the data in {@link #buffers} to the channel.
+ * It must be called before committing a PutBlock in the middle of the
stream;
+ * otherwise, the buffered data is not yet in the block file.
+ */
+ public void drainBuffers() throws IOException {
+ assertOpen();
+ if (!buffers.isEmpty()) {
+ pollAllAndWrite(buffers, super::writeFileChannel, b -> null);
}
}
Review Comment:
I think we can just make drainbuffer to sth like:
```java
public void drainBuffers() throws IOException {
assertOpen();
final ReferenceCountedObject<ByteBuf> ref = buffers.pollAll();
try {
writeFully(ref.retain().nioBuffer(), super::writeFileChannel);
} finally {
ref.release();
}
}
```
I think the reuse attempt here is a little bit not that general.
##########
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/impl/Buffers.java:
##########
Review Comment:
non block:
thinking we dont need to throw if buffer is empty, that would make caller
easier to use. (they dont need to consider whether to check `isEmpty()` or not.)
```suggestion
ReferenceCountedObject<ByteBuf> pollAll() {
if (deque.isEmpty()) {
return ReferenceCountedObject.wrap(Unpooled.EMPTY_BUFFER);
}
```
##########
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/impl/KeyValueStreamDataChannel.java:
##########
@@ -141,30 +142,45 @@ protected void cleanupInternal() throws IOException {
* Note that the PutBlock proto at the end is ignored; see HDDS-12007.
*/
private void writeBuffers() throws IOException {
- final ReferenceCountedObject<ByteBuf> ref = buffers.pollAll();
- final ByteBuf buf = ref.retain();
- try {
- setEndIndex(buf);
- // write the remaining data
- writeFully(buf.nioBuffer(), super::writeFileChannel);
- } finally {
- ref.release();
+ pollAllAndWrite(buffers, super::writeFileChannel, b -> {
+ setEndIndex(b);
+ return null;
+ });
+ }
+
+ /**
+ * Write all the data in {@link #buffers} to the channel.
+ * It must be called before committing a PutBlock in the middle of the
stream;
+ * otherwise, the buffered data is not yet in the block file.
+ */
+ public void drainBuffers() throws IOException {
+ assertOpen();
+ if (!buffers.isEmpty()) {
+ pollAllAndWrite(buffers, super::writeFileChannel, b -> null);
}
}
Review Comment:
non block:
I think we can just make drainbuffer to sth like:
```java
public void drainBuffers() throws IOException {
assertOpen();
final ReferenceCountedObject<ByteBuf> ref = buffers.pollAll();
try {
writeFully(ref.retain().nioBuffer(), super::writeFileChannel);
} finally {
ref.release();
}
}
```
I think the reuse attempt here is a little bit not that general.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
