Hi Lei, I don't know any NiFi built-in feature to achieve that. To distribute CaptureChangeMySQL load among nodes, I'd deploy separate standalone NiFi (or even MiNiFi Java) in addition to the main NiFi cluster for the main data flow.
For example, if there are 5 databases and 3 NiFi nodes, deploy a 3 node NiFi cluster with an InputPort. And also, run standalone NiFi/MiNiFi processes on each node, too, node-a (datasource 1 and 2), node-b (datasource 3 and 4), node-c (datasource 5) then use RemoteProcessGroup to send captured data to the main NiFi cluster. This approach may be harder to maintain, but feasible. Thanks, Koji On Wed, Oct 9, 2019 at 3:06 PM [email protected] <[email protected]> wrote: > > I am using CaptureChangeMySQL to replicate the database. > There are many data sources and so there're many CaptureChangeMySQL > processors. > The CaptureChangeMySQL throws same slave id error if scheduled on all nodes. > So it can only be scheduled on primary node. This causes very heavy load on > the primary node. > > Is there any method than i can distribute the CaptureChangeMySQL processors > to all nodes instead of only to primary node? > > Thanks, > Lei > > ________________________________ > [email protected]
