[
https://issues.apache.org/jira/browse/NIFI-6760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16948637#comment-16948637
]
ASF subversion and git services commented on NIFI-6760:
-------------------------------------------------------
Commit d64f4fa942342482012b3caf4f8e3bf475685d03 in nifi's branch
refs/heads/master from markap14
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=d64f4fa ]
NIFI-6760: When writing/reading the length of a DataFrame, do so usin… (#3801)
* NIFI-6760: When writing/reading the length of a DataFrame, do so using a
4-byte integer instead of a 2-byte short. When using uncompressed data, we know
that the length of the DataFrame will be no more than 64 KB so a 2-byte short
is sufficient. However, if data is compresed, there's a chance that the
compressed form of the data will be larger than the uncompressed form (for
example, with random binary data or with encrypted data). In this situation, we
can end up overflowing the 2-byte short, which causes the length that is
written to be wrong. Using a 4-byte integer avoids this situation.
* NIFI-6760: Fixed unit tests
> Load-Balanced Connections can error if configured to compress content
> ---------------------------------------------------------------------
>
> Key: NIFI-6760
> URL: https://issues.apache.org/jira/browse/NIFI-6760
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
> Reporter: Mark Payne
> Assignee: Mark Payne
> Priority: Major
> Fix For: 1.10.0
>
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> Using load-balanced connections, I can sometimes see failures when data is
> being compressed. On the client side, I see in the logs:
> {code:java}
> 2019-10-09 15:19:40,688 ERROR [Load-Balanced Client Thread-4]
> o.a.n.c.q.c.c.a.n.NioAsyncLoadBalanceClient Failed to communicate with Peer
> localhost:5672
> java.io.IOException: Broken pipe
> at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
> at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47)
> at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93)
> at sun.nio.ch.IOUtil.write(IOUtil.java:65)
> at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:471)
> at
> org.apache.nifi.controller.queue.clustered.client.async.nio.PeerChannel.write(PeerChannel.java:132)
> at
> org.apache.nifi.controller.queue.clustered.client.async.nio.LoadBalanceSession.communicate(LoadBalanceSession.java:165)
> at
> org.apache.nifi.controller.queue.clustered.client.async.nio.NioAsyncLoadBalanceClient.communicate(NioAsyncLoadBalanceClient.java:242)
> at
> org.apache.nifi.controller.queue.clustered.client.async.nio.NioAsyncLoadBalanceClientTask.run(NioAsyncLoadBalanceClientTask.java:76)
> at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> While on the server side I see:
> {code:java}
> 2019-10-09 15:19:40,687 ERROR [Load-Balance Server Thread-4]
> o.a.n.c.q.c.s.ConnectionLoadBalanceServer Failed to communicate with Peer
> /127.0.0.1:50895
> java.io.EOFException: Unexpected end of ZLIB input stream
> at java.util.zip.InflaterInputStream.fill(InflaterInputStream.java:240)
> at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:158)
> at java.util.zip.GZIPInputStream.read(GZIPInputStream.java:117)
> at org.apache.nifi.stream.io.StreamUtils.fillBuffer(StreamUtils.java:89)
> at
> org.apache.nifi.controller.queue.clustered.server.StandardLoadBalanceProtocol.consumeContent(StandardLoadBalanceProtocol.java:512)
> at
> org.apache.nifi.controller.queue.clustered.server.StandardLoadBalanceProtocol.receiveFlowFile(StandardLoadBalanceProtocol.java:445)
> at
> org.apache.nifi.controller.queue.clustered.server.StandardLoadBalanceProtocol.receiveFlowFiles(StandardLoadBalanceProtocol.java:252)
> at
> org.apache.nifi.controller.queue.clustered.server.StandardLoadBalanceProtocol.receiveFlowFiles(StandardLoadBalanceProtocol.java:133)
> at
> org.apache.nifi.controller.queue.clustered.server.ConnectionLoadBalanceServer$CommunicateAction.run(ConnectionLoadBalanceServer.java:152)
> at java.lang.Thread.run(Thread.java:745)
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
