The approach we use for disaster recovery might be a start.
* nifi (user) has a cron job that runs BASH script on PROD server node
which copies the flow.xml.gz file to be used on another cluster to a temp
folder.
* The script decrypts the current passwords/properties in flow.xml.gz
and encrypts with new BACKUP's password which re-encrypts sensitive properties.
*
/var/lib/ambari-agent/tmp/nifi-toolkit-1.5.0.3.1.0.0-564/bin/encrypt-config.sh
-b /usr/hdf/current/nifi/conf/bootstrap.conf -f
/apps/var/lib/nifi/conf/flow.xml.gz -g ~/flow.xml.gz -n
/usr/hdf/current/nifi/conf/nifi.properties -o
/usr/hdf/current/nifi/conf/backup/nifi.properties -s $#$#$#$#$#$#$#$ -x
* Script also unzips flow.xml.gz, makes modifications by calling python
script that changes a few labels and parameters using xml.etree.
* The flow.xml.gz is zipped and copied to /tmp with chmod changes
* Another cron copies the flow.xml.gz to the BACKUP servers offsite.
* Cron on BACKUP servers runs script that stops NiFi, copies new
flow.xml.gz overwriting existing one. Restarts NiFi with all processors
stopped. Otherwise system starts with flows in last known state in PROD.
Caveats are:
NiFi versions and configurations the same
Users should be the same
Should point to same NiFi Registry, if used
If using Ranger, run a script exporting configuration, renaming it and sending
it to BACKUP's Ranger
User authorization and authentication mechanisms the same.
Note that the password for encrypting flow for the BACKUP server is exposed in
script ($#$#$#$#$#$#$#$).
I don't remember what else...
Mike
From: Andy LoPresto [mailto:[email protected]]
Sent: Monday, October 28, 2019 12:09 PM
To: [email protected]
Subject: Re: How to deploy NiFi processors change to multiple NiFi instances?
I would generally want a human intervention step in this process, but if you
want this to be fully-automated, you can script that using the Event Hooks [1]
feature. As Edward noted, the NiFi Toolkit or NiPyAPI could be invoked from
here to execute whatever steps you want taken.
[1]
https://nifi.apache.org/docs/nifi-registry-docs/html/administration-guide.html#event-hooks
Andy LoPresto
[email protected]<mailto:[email protected]>
[email protected]
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69
On Oct 28, 2019, at 1:38 AM, Edward Armes
<[email protected]<mailto:[email protected]>> wrote:
Hi Lei,
As far as I'm aware there currently isn't an out of the box method to do this
However I would look at using a combination of the nifi registry and tools in
nifi toolkit to roll your own.
Also, there have been multiple questions to both this and the dev mailing list,
asking what could be done. I would also suggest searching the list archives and
see what others did and didn't do as well.
Edward
On Mon, 28 Oct 2019, 08:14
[email protected]<mailto:[email protected]>,
<[email protected]<mailto:[email protected]>> wrote:
We have many standalone NiFi instances and all running the same NiFi Flow.
If the flow changes, how to deploy the change to all NiFi instances
automatically?
Thanks,
Lei
________________________________
[email protected]<mailto:[email protected]>
