Hello Oleg,
Many thanks for your reply! I remembered having diff'ed NiFi templates in the past and that was a bit messy. However, this was certainly before NiFi 1.0, so from a quick look it appears that things are better now in that regard. Thanks for the hint! However, the situation with moving the controller services associated with a processor seems still problematic in my humble opinion. Let's say that in my source VM, I have created a Database and an HTTPContext map service at the top level of my canvas. These are being used inside many different processors that belong to different process groups. Now, if I export my canvas as a template and import it on another VM, I don't see these services at the top level of my canvas on my target VM. The services are being "compartmentalised" into the process groups and sub process groups of my template. Therefore, if I have to do a change (say the port of the database is different) I need to go through all of them manually as a change in a "compartmentalised" service is not reflected on another process group. An alternative, which I have been using, is to recreate the service at the top level of the canvas on the target VM, but this means that I need to go through all the processors of the template I moved and associate them with the new top level service of my target VM, which is very error-prone in an CI/CD pipeline. Isn't there an easier way to migrate templates with their associated services? Many thanks again, Panos Ps. writing all this, it came to mind that I could include all my canvas in one process group in my source VM, generate my service inside that and then move all of that in my target VM. Not ideal, but should work [😉] ________________________________ From: Oleg Zhurakousky <[email protected]> Sent: Friday, January 13, 2017 1:18 PM To: [email protected] Subject: Re: Migrating NiFi templates/canvas Panos What version of NiFi are you using? The issue with the NiFi templates diffs has actually been addressed (see https://issues.apache.org/jira/browse/NIFI-826). So as of NiFi 1.0 you can export templates in a deterministic way and then execute diff on them to see only what was changed. It was addressed specifically for the SDLC issue you are describing. Also, when processor is associated with controller service and you created a template that includes such processor, its dependent controller service will end up on the template and back in your environment once such template is imported. [NIFI-826] Export templates in a deterministic way - ASF JIRA<https://issues.apache.org/jira/browse/NIFI-826> issues.apache.org Templates should be exported in a deterministic way so that they can be compared or diff'ed with another. Items to consider... The ordering of components The id's ... Anyway, give it a shot and let us know. Cheers Oleg On Jan 13, 2017, at 4:49 AM, Panos Geo <[email protected]<mailto:[email protected]>> wrote: Hello all, We have a Continuous Integration/Continuous Development pipeline that involves for each stage (development, testing, customer deployment) a dedicated virtual machine running a NiFi instance. As such, for each stage of the pipeline we have to migrate the changes of our NiFi canvas from one virtual machine to another. For this we encounter two big problems : 1. As far as I know, there is no easy way to diff NiFi canvases (or templates) so that we can recognize the changes and don’t have to export and import the full canvas each time (as there are configuration changes in the target VM that we wouldn’t like to overwrite). 2. If we have to export the full canvas, is there a way to reassociate processors with the required services on the target virtual machine (e.g. DBCPConnectionPool or HTTPContextMap) without having to go through each processor explicitly and do the reassignment? Many thanks in advance, Panos
