Panos, The current behavior is that Controller Services are only included if they are referenced by a component in the template. While this was initially by designed, we have received comments about it from folks looking to export their entire configuration. There is an outstanding JIRA [1] to update this behavior.
Matt [1] https://issues.apache.org/jira/browse/NIFI-2895 On Fri, Jan 13, 2017 at 12:01 PM, Panos Geo <[email protected]> wrote: > Having just checked the idea in my PS, it doesn't seem to work as the > transferred controller services are at a process group level and hence are " > compartmentalised" after the migration... > > > Any thoughts on this would be highly appreciated. > > > Many thanks, > > Panos > > > ------------------------------ > *From:* Panos Geo <[email protected]> > *Sent:* Friday, January 13, 2017 4:47 PM > > *To:* [email protected] > *Subject:* Re: Migrating NiFi templates/canvas > > > Hello Oleg, > > > Many thanks for your reply! > > > I remembered having diff'ed NiFi templates in the past and that was a bit > messy. However, this was certainly before NiFi 1.0, so from a quick look > it appears that things are better now in that regard. Thanks for the hint! > > > However, the situation with moving the controller services associated with > a processor seems still problematic in my humble opinion. Let's say that > in my source VM, I have created a Database and an HTTPContext map service > at the *top level* of my canvas. These are being used inside many > different processors that belong to different process groups. Now, if I > export my canvas as a template and import it on another VM, I don't see > these services at the *top level* of my canvas on my target VM. The > services are being "compartmentalised" into the process groups and sub > process groups of my template. Therefore, if I have to do a change (say > the port of the database is different) I need to go through all of them > manually as a change in a "compartmentalised" service is not reflected on > another process group. > > > An alternative, which I have been using, is to recreate the service at > the top level of the canvas on the target VM, but this means that I need to > go through all the processors of the template I moved and associate them > with the new top level service of my target VM, which is very error-prone > in an CI/CD pipeline. > > > Isn't there an easier way to migrate templates with their associated > services? > > > Many thanks again, > > Panos > > > > Ps. writing all this, it came to mind that I could include all my canvas > in one process group in my source VM, generate my service inside that and > then move all of that in my target VM. Not ideal, but should work [image: > 😉] > > > ------------------------------ > *From:* Oleg Zhurakousky <[email protected]> > *Sent:* Friday, January 13, 2017 1:18 PM > *To:* [email protected] > *Subject:* Re: Migrating NiFi templates/canvas > > Panos > > What version of NiFi are you using? > The issue with the NiFi templates diffs has actually been addressed (see > https://issues.apache.org/jira/browse/NIFI-826). So as of NiFi 1.0 you > can export templates in a deterministic way and then execute diff on them > to see only what was changed. It was addressed specifically for the SDLC > issue you are describing. Also, when processor is associated with > controller service and you created a template that includes such processor, > its dependent controller service will end up on the template and back in > your environment once such template is imported. > [NIFI-826] Export templates in a deterministic way - ASF JIRA > <https://issues.apache.org/jira/browse/NIFI-826> > issues.apache.org > Templates should be exported in a deterministic way so that they can be > compared or diff'ed with another. Items to consider... The ordering of > components The id's ... > > > Anyway, give it a shot and let us know. > Cheers > Oleg > > On Jan 13, 2017, at 4:49 AM, Panos Geo <[email protected]> wrote: > > Hello all, > > We have a Continuous Integration/Continuous Development pipeline that > involves for each stage (development, testing, customer deployment) a > dedicated virtual machine running a NiFi instance. As such, for each stage > of the pipeline we have to migrate the changes of our NiFi canvas from one > virtual machine to another. > > For this we encounter two big problems : > > 1. As far as I know, there is no easy way to diff NiFi canvases (or > templates) so that we can recognize the changes and don’t have to > export and import the full canvas each time (as there are configuration > changes in the target VM that we wouldn’t like to overwrite). > 2. If we have to export the full canvas, is there a way to reassociate > processors with the required services on the target virtual machine (e.g. > DBCPConnectionPool or HTTPContextMap) without having to go through each > processor explicitly and do the reassignment? > > > Many thanks in advance, > Panos > > >
