markap14 commented on a change in pull request #3801: NIFI-6760: When
writing/reading the length of a DataFrame, do so usin…
URL: https://github.com/apache/nifi/pull/3801#discussion_r333535006
##########
File path:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/queue/clustered/client/async/nio/LoadBalanceSession.java
##########
@@ -319,16 +319,15 @@ private ByteBuffer getFlowFileContent() throws
IOException {
final byte[] compressed = compressDataFrame(byteBuffer,
bytesRead);
final int compressedMaxLen = compressed.length;
- buffer = ByteBuffer.allocate(3 + compressedMaxLen);
+ buffer = ByteBuffer.allocate(5 + compressedMaxLen);
buffer.put((byte)
LoadBalanceProtocolConstants.DATA_FRAME_FOLLOWS);
- buffer.putShort((short) compressedMaxLen);
Review comment:
The short was used because we knew that the data frame wouldn't be larger
than 64 KB, so an unsigned short is sufficient for the length. The problem is
that after compressing the content of the frame, if the data in the frame does
not compress well, the size can actually exceed 64 KB, so the unsigned short is
not large enough to convey the actual size. So it needed to be enlarged to a
32-bit int instead.
It is true that the wire format changes here. However, that should be okay
because NiFi does not currently support running a cluster where different nodes
are running different versions of nifi - they must all be the same version.
Since this is used only for communication between nodes in the same cluster, we
know that they all will be running the same version & hence the same wire
format will be used.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
