Panos, I'm not sure what version of NiFi you're using but in 1.1.1 another issue [1] was addressed that may be what you're seeing now. If you're still on 1.0.0 or 1.1.0 maybe try upgrading to 1.1.1.
Matt [1] https://issues.apache.org/jira/browse/NIFI-3173 On Fri, Jan 13, 2017 at 1:51 PM, Panos Geo <[email protected]> wrote: > Hello Matt, > > > Thanks for your reply. > > > I fully get that, but my requirement is in addition to this. The > (top) controller service is being referenced by multiple processors in my > template, which however belong in different sub process groups. When I > import this template in another NiFi instance this controller service is > present, but it is being "split" to different process group controller > services, which I cannot configure centrally (as I would have done if that > was a top level controller service as it was in the original NiFi instance > before exporting the template). > > > Many thanks, > > Panos > > > ------------------------------ > *From:* Matt Gilman <[email protected]> > *Sent:* Friday, January 13, 2017 6:27 PM > > *To:* [email protected] > *Subject:* Re: Migrating NiFi templates/canvas > > Panos, > > The current behavior is that Controller Services are only included if they > are referenced by a component in the template. While this was initially by > designed, we have received comments about it from folks looking to export > their entire configuration. There is an outstanding JIRA [1] to update this > behavior. > > Matt > > [1] https://issues.apache.org/jira/browse/NIFI-2895 > > On Fri, Jan 13, 2017 at 12:01 PM, Panos Geo <[email protected]> > wrote: > >> Having just checked the idea in my PS, it doesn't seem to work as the >> transferred controller services are at a process group level and hence are " >> compartmentalised" after the migration... >> >> >> Any thoughts on this would be highly appreciated. >> >> >> Many thanks, >> >> Panos >> >> >> ------------------------------ >> *From:* Panos Geo <[email protected]> >> *Sent:* Friday, January 13, 2017 4:47 PM >> >> *To:* [email protected] >> *Subject:* Re: Migrating NiFi templates/canvas >> >> >> Hello Oleg, >> >> >> Many thanks for your reply! >> >> >> I remembered having diff'ed NiFi templates in the past and that was a >> bit messy. However, this was certainly before NiFi 1.0, so from a quick >> look it appears that things are better now in that regard. Thanks for >> the hint! >> >> >> However, the situation with moving the controller services associated >> with a processor seems still problematic in my humble opinion. Let's say >> that in my source VM, I have created a Database and an HTTPContext map >> service at the *top level* of my canvas. These are being used inside >> many different processors that belong to different process groups. Now, >> if I export my canvas as a template and import it on another VM, I don't >> see these services at the *top level* of my canvas on my target VM. The >> services are being "compartmentalised" into the process groups and sub >> process groups of my template. Therefore, if I have to do a change (say >> the port of the database is different) I need to go through all of them >> manually as a change in a "compartmentalised" service is not reflected >> on another process group. >> >> >> An alternative, which I have been using, is to recreate the service at >> the top level of the canvas on the target VM, but this means that I need to >> go through all the processors of the template I moved and associate them >> with the new top level service of my target VM, which is very error-prone >> in an CI/CD pipeline. >> >> >> Isn't there an easier way to migrate templates with their associated >> services? >> >> >> Many thanks again, >> >> Panos >> >> >> >> Ps. writing all this, it came to mind that I could include all my canvas >> in one process group in my source VM, generate my service inside that and >> then move all of that in my target VM. Not ideal, but should work [image: >> 😉] >> >> >> ------------------------------ >> *From:* Oleg Zhurakousky <[email protected]> >> *Sent:* Friday, January 13, 2017 1:18 PM >> *To:* [email protected] >> *Subject:* Re: Migrating NiFi templates/canvas >> >> Panos >> >> What version of NiFi are you using? >> The issue with the NiFi templates diffs has actually been addressed (see >> https://issues.apache.org/jira/browse/NIFI-826). So as of NiFi 1.0 you >> can export templates in a deterministic way and then execute diff on them >> to see only what was changed. It was addressed specifically for the SDLC >> issue you are describing. Also, when processor is associated with >> controller service and you created a template that includes such processor, >> its dependent controller service will end up on the template and back in >> your environment once such template is imported. >> [NIFI-826] Export templates in a deterministic way - ASF JIRA >> <https://issues.apache.org/jira/browse/NIFI-826> >> issues.apache.org >> Templates should be exported in a deterministic way so that they can be >> compared or diff'ed with another. Items to consider... The ordering of >> components The id's ... >> >> >> Anyway, give it a shot and let us know. >> Cheers >> Oleg >> >> On Jan 13, 2017, at 4:49 AM, Panos Geo <[email protected]> >> wrote: >> >> Hello all, >> >> We have a Continuous Integration/Continuous Development pipeline that >> involves for each stage (development, testing, customer deployment) a >> dedicated virtual machine running a NiFi instance. As such, for each stage >> of the pipeline we have to migrate the changes of our NiFi canvas from one >> virtual machine to another. >> >> For this we encounter two big problems : >> >> 1. As far as I know, there is no easy way to diff NiFi canvases (or >> templates) so that we can recognize the changes and don’t have to >> export and import the full canvas each time (as there are configuration >> changes in the target VM that we wouldn’t like to overwrite). >> 2. If we have to export the full canvas, is there a way to >> reassociate processors with the required services on the target virtual >> machine (e.g. DBCPConnectionPool or HTTPContextMap) without having to go >> through each processor explicitly and do the reassignment? >> >> >> Many thanks in advance, >> Panos >> >> >> >
