Thanks， got it
[email protected] From: Purushotham Pushpavanthar Date: 2019-07-30 04:10 To: users Subject: Re: Re: CaptureChangeMySQL throws ServerException when configured with all nodes execution Yes, CaptureChangeMySQL processor is designed run on Primary node which gives HA to the processor in a cluster. The CaptureChangeMySQL runs on a single thread but consumes JVM memory based on load from source MySQL server. Running multiple CaptureChangeMySQL processors connecting to different MySQL servers shouldn't be a problem if the buffer queue size and queue offer timeouts are configured correctly. No, there is no other way to configure them to run on different nodes in a cluster. If that is what you want, then you can have one standalone node running for each CaptureChangeMySQL processor. Regards, Purushotham Pushpavanth On Sun, 28 Jul 2019 at 16:08, [email protected] <[email protected]> wrote: So the CaptureChangeMySQL processor is designed to be configured as run on Primary node? even my nifi is cluster mode. Now I have many CaptureChangeMySQL processors, connecting to different Mysql Server. If all run on the primary node, the primary node is too high loaded. Is there any method to distribute the different CaptureChangeMySQL processors to different nodes? Thanks [email protected] From: Purushotham Pushpavanthar Date: 2019-07-28 16:33 To: users Subject: Re: CaptureChangeMySQL throws ServerException when configured with all nodes execution Hi, If CaptureChangeMySQL is configured to run on all nodes, there will duplicate events for sure. However, in your case the exception is thrown for having same "Sever ID" for all instances of the processor connecting to MySQL server. I don't see any data loss due currecurrent execution. Regards, Purushotham Pushpavanth On Fri, 26 Jul 2019 at 08:07, [email protected] <[email protected]> wrote: If I config the Scheduling Execution to All nodes, there will be Exception in the log: com.github.shyiko.mysql.binlog.network.ServerException: A slave with the same server_uuid/server_id as this slave has conne cted to the master; the first event 'mysql-bin.000502' at 298034601, the last event read from './mysql-bin.000502' at 29865 2073, the last byte read from './mysql-bin.000502' at 298652073. at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:882) at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:559) at com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:793) at java.lang.Thread.run(Thread.java:745) Does it affect the accuracy of the CaptureChangeMySQL result? [email protected]
