Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22105#discussion_r210134223
--- Diff:
common/network-common/src/main/java/org/apache/spark/network/protocol/MessageWithHeader.java
---
@@ -140,8 +140,24 @@ private int copyByteBuf(ByteBuf buf,
WritableByteChannel target) throws IOExcept
// SPARK-24578: cap the sub-region's size of returned nio buffer to
improve the performance
// for the case that the passed-in buffer has too many components.
int length = Math.min(buf.readableBytes(), NIO_BUFFER_LIMIT);
--- End diff --
My understanding is that the code here is avoiding the copy by using
`nioBuffers()` already, so that limit shouldn't be needed, right? Or maybe I'm
missing something.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
