Thanks! Sorry for the delay! What do you mean by creating microclusters? Is this a Nifi functionnality (ie. hierarchical cluster of clusters) ? Or is it a cluster design that must be created thoroughly?
Benoit. Le 26/11/2016 à 02:13, Andy LoPresto a écrit : > Benoit, > > Every NiFi node can run many disparate flows, and you can separate > them into distinct process groups for logical divisions. > > You can trigger these flows in a few ways. The first, you could set > the first processor in the flow to be triggered by event receipt > rather than schedule driven so that it doesn't run until a unit of > data enters the flow. The other thing you could do is use the REST API > to enable/disable individual processors/flows and trigger that API > invocation via cron or some other scheduling system external to NiFi. > Finally, you could use an ExecuteScript processor to perform more > complex custom logic to determine which flows should be running at any > given time (and then use the REST API to enable/disable them). > > If you really find that these changes cannot be synchronized across > the single flow residing on a NiFi cluster, you could segment and > subdivide your cluster into microclusters where each shares a partial > flow (grouped by performance considerations or logical concepts). > > Andy LoPresto > [email protected] <mailto:[email protected]> > [email protected] <mailto:[email protected]> > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > > On Nov 25, 2016, at 04:13, De Mezzo Benoit <[email protected] > <mailto:[email protected]>> wrote: > >> Hi all, >> >> Here is my problem: I need to run many small etl flow (say a thousand) >> at flow-dedicated times. As theses flows may be added, removed, changed >> all the time, they can not be easily merged within a big-meta flow to be >> run in the Nifi cluster. >> >> I would like to know if there a way to run multiple flows within a nifi >> cluster? >> >> Or may be there is a way to generate, per flow, a small standalone >> binary (jar?) to run only once a flow against an embedded Nifi engine >> (each jar will be managed by a task scheduling cluster) ? >> >> Thanks ! >> >> Benoit. >>
