Yes, CaptureChangeMySQL processor is designed run on Primary node which gives HA to the processor in a cluster. The CaptureChangeMySQL runs on a single thread but consumes JVM memory based on load from source MySQL server. Running multiple CaptureChangeMySQL processors connecting to different MySQL servers shouldn't be a problem if the buffer queue size and queue offer timeouts are configured correctly.
No, there is no other way to configure them to run on different nodes in a cluster. If that is what you want, then you can have one standalone node running for each CaptureChangeMySQL processor. Regards, Purushotham Pushpavanth On Sun, 28 Jul 2019 at 16:08, [email protected] < [email protected]> wrote: > > So the CaptureChangeMySQL processor is designed to be configured as run > on Primary node? even my nifi is cluster mode. > Now I have many CaptureChangeMySQL processors, connecting to > different Mysql Server. If all run on the primary node, the primary node > is too high loaded. > Is there any method to distribute the different CaptureChangeMySQL > processors to different nodes? > > Thanks > ------------------------------ > [email protected] > > > *From:* Purushotham Pushpavanthar <[email protected]> > *Date:* 2019-07-28 16:33 > *To:* users <[email protected]> > *Subject:* Re: CaptureChangeMySQL throws ServerException when configured > with all nodes execution > Hi, > > If CaptureChangeMySQL is configured to run on all nodes, there will > duplicate events for sure. > However, in your case the exception is thrown for having same "Sever ID" > for all instances of the processor connecting to MySQL server. > I don't see any data loss due currecurrent execution. > > Regards, > Purushotham Pushpavanth > > > > On Fri, 26 Jul 2019 at 08:07, [email protected] < > [email protected]> wrote: > >> >> If I config the Scheduling Execution to All nodes, there will be >> Exception in the log: >> >> com.github.shyiko.mysql.binlog.network.ServerException: A slave with the >> same server_uuid/server_id as this slave has conne >> cted to the master; the first event 'mysql-bin.000502' at 298034601, the >> last event read from './mysql-bin.000502' at 29865 >> 2073, the last byte read from './mysql-bin.000502' at 298652073. >> at >> com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:882) >> at >> com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:559) >> at >> com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:793) >> at java.lang.Thread.run(Thread.java:745) >> >> Does it affect the accuracy of the CaptureChangeMySQL result? >> >> ------------------------------ >> [email protected] >> >
