Github user liyezhang556520 commented on a diff in the pull request:
https://github.com/apache/spark/pull/12038#discussion_r57742478
--- Diff:
common/network-common/src/main/java/org/apache/spark/network/util/TransportFrameDecoder.java
---
@@ -139,14 +139,18 @@ private ByteBuf decodeNext() throws Exception {
return nextBufferForFrame(remaining);
}
- // Otherwise, create a composite buffer.
- CompositeByteBuf frame = buffers.getFirst().alloc().compositeBuffer();
+ // Otherwise, create a new buffer to hold the data of all frame splits.
+ ByteBuf frame = buffers.getFirst().alloc().buffer(remaining,
remaining);
--- End diff --
here allocate a new `ByteBuf`, it will result in one time copy for each
netty frame, exactly one time copy operation is much more acceptable compared
with too many copies of `compositeBuffer`.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
