Hi Ashwin, The embedded ZooKeeper is provided as a convenience so that you can easily test running things in a cluster on a laptop, etc. However, it struggles when your nodes are handling any kind of significant data rate. It is always recommended that an external ZooKeeper be used for any sort of production use.
Thanks -Mark > On Sep 25, 2018, at 6:03 AM, ashwin konale <[email protected]> wrote: > > Hi, > > I am running 3 node nifi cluster (embedded zookeeper) with following flow. > It pulls the data from mysql binlogs using CaptureChangeMysql and sending > it to HDFS. > > CDC[Primary node only] -> outputPort -> RPG > > inputPort -> MergeContent -> PutHdfs > > I am using RPG to distribute load among other nodes in cluster. CDC reads > large amount of data (~10 Mbps). But as soon as I start CDC processor, nifi > ui hangs, and I get "javax.ws.rs.ProcessingException: > java.net.SocketTimeoutException: Read timed out" , I have gone through > various threads and tweaked certain properties like, > > nifi.cluster.node.connection.timeout = 30 seconds > nifi.cluster.node.read.timeout = 30 sec > nifi.cluster.node.protocol.threads = 20 > nifi.web.jetty.threads = 400 > > Nothing unusual in the logs except data flow seems to be continuing. I am > not able to figure out what is going wrong. Can someone please help me with > it. Let me know what more information I need to provide in order to debug > this. > > Stacktrace: > > 2018-09-25 12:00:49,888 ERROR [LearnerHandler-/10.10.10.10:31955] > o.a.z.server.quorum.LearnerHandler Unexpected exception causing shutdown > while sock still open > > java.net.SocketTimeoutException: Read timed out > > at java.net.SocketInputStream.socketRead0(Native Method) > > at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) > > at java.net.SocketInputStream.read(SocketInputStream.java:171) > > at java.net.SocketInputStream.read(SocketInputStream.java:141) > > at java.io.BufferedInputStream.fill(BufferedInputStream.java:246) > > at java.io.BufferedInputStream.read(BufferedInputStream.java:265) > > at java.io.DataInputStream.readInt(DataInputStream.java:387) > > at org.apache.jute.BinaryInputArchive.readInt(BinaryInputArchive.java:63) > > at > org.apache.zookeeper.server.quorum.QuorumPacket.deserialize(QuorumPacket.java:83) > > at > org.apache.jute.BinaryInputArchive.readRecord(BinaryInputArchive.java:103) > > at > org.apache.zookeeper.server.quorum.LearnerHandler.run(LearnerHandler.java:546) > > 2018-09-25 12:00:51,901 INFO [Process Cluster Protocol Request-7] > o.a.n.c.p.impl.SocketProtocolListener Finished processing request > e181bdbd-00c7-4f3b-b69a-34ff7c1c9cee (type=HEARTBEAT, length=3565 bytes) > from nifiapp-1002:8080 in 1 millis > > > Any help would be much appreciated. > Ashwin
