Hi, In our big data environment one of the architectural principles is to schedule jobs with Azure Automation (runbooks). A scheduling database is used to decide when to start which jobs. NiFi flows however are currently being scheduled in NiFi itself. We're looking for a good approach to move this over to runbooks. I see a couple of options:
* Have each flow start with a timer driven processor, where the run schedule is an hour or so. This processor will be stopped by default, and can be turned on via the API. It is then stopped at some point before the run schedule ends, preventing the processor from running twice. * Use a ListenHTTP processor that we can POST a message to that specifies which flow to start. Do something like RouteOnAttribute to choose the right flow. I imagine this as being one ListenHTTP processor that is connected to all flows. * Translate the schedule from the scheduling database to a ChronTrigger expression. Check if the CRON schedule on the processor is indeed set to that schedule. If not, stop the processor, change the schedule and start it again. If it is, do nothing and assume it'll run. This one seems convoluted on the one hand, but requires the least architecture within NiFi itself I imagine. What do you think? Has anyone had to deal with something like this? How did you solve it? I can't find much information about this on the web, although I could be using the wrong terms. Kind regards, Walter Vos ________________________________ Deze e-mail, inclusief eventuele bijlagen, is uitsluitend bestemd voor (gebruik door) de geadresseerde. De e-mail kan persoonlijke of vertrouwelijke informatie bevatten. Openbaarmaking, vermenigvuldiging, verspreiding en/of verstrekking van (de inhoud van) deze e-mail (en eventuele bijlagen) aan derden is uitdrukkelijk niet toegestaan. Indien u niet de bedoelde geadresseerde bent, wordt u vriendelijk verzocht degene die de e-mail verzond hiervan direct op de hoogte te brengen en de e-mail (en eventuele bijlagen) te vernietigen. Informatie vennootschap<http://www.ns.nl/emaildisclaimer>
