Manoj, That is a very good point, and it is something that we are working toward. However, it does get a little bit more complicated than this. If you have some Processor, say Processor A running on some arbitrary node, there will often be times that you will also need another Processor, Processor B, running on that same node.
Using a Primary Node means that we are able to accomplish this easily, but as you are noting here, it is quite limiting. In version 1.0.0 of NiFi, one of the big changes in a Zero-Master clustering design, whereby the Primary Node is automatically elected and fails over to a different node whenever the Primary Node leaves the cluster. This improves the overall functionality of Primary Node but does not address the issue here, of avoiding scheduling all "singleton" processors on the same node. I think the path that we'd like to take moving forward, post-1.0.0, is to provide a mechanism that allows the user to schedule a Processor to run in some sort of named "Scheduling Group". So, for instance, you could say Processor A and B should both run in "Group A" but Processor C should run in "Group C". This way, we can ensure that Processors that need to run together can do so while at the same time avoiding the need for all such processors to run on the same node. Does this sound like a reasonable approach for your use case? Thanks -Mark > On Apr 5, 2016, at 3:08 AM, <[email protected]> > <[email protected]> wrote: > > For the purposes of symmetry of the NiFi Cluster, and so that the initial > ingest of content is not limited to just one primary node in the NiFi > cluster, would it not be beneficial for the framework to have the ability to > schedule an Isolated Processor on ANY ONE of available nodes in the NiFi > Cluster? > > Regards, Manoj > > Manoj Seshan - Senior Architect > Platform Content Technology, Bangalore > > Voice: +91-9686578756 +91-80-67492572
