You can do nested versioning since the beginning in 0.1.0. A common scenario might be to have several teams build different versioned flows, and then someone who is in charge of deploying them will create another version PG that combines the nested versioned process groups of each of these teams.
The outer versioned flow in registry does not fully contain the others, just pointers to the actual versioned flows, which technically could come from another registry if desired. On Tue, Jul 10, 2018 at 11:22 AM, Boris Tyukin <bo...@boristyukin.com> wrote: > thanks Bryan. I saw your blog post on that. I think with registry 0.1 it was > not possible to version nested PGs within parent PGs so I could not have > "templatized" PG which has it is own version and use that PG with other > versioned PGs. Has it changed with registry 0.2 now out? > > On Tue, Jul 10, 2018 at 11:08 AM Bryan Bende <bbe...@gmail.com> wrote: >> >> Boris, >> >> Regarding templates being limited... templates were really made as a >> way to share example flows, or help with debugging if you need to send >> someone your flow. Unfortunately they turned into a deployment >> mechanism since there wasn't a better solution at the time. >> >> Using NiFi Registry should now be the preferred solution, and you can >> sync changes in-place. You can have many versioned process groups tied >> to the same versioned flow in a registry and update them all. >> >> Thanks, >> >> Bryan >> >> >> On Tue, Jul 10, 2018 at 10:50 AM, Boris Tyukin <bo...@boristyukin.com> >> wrote: >> > I like Ed's recommendations and doing something similar. I use ISPs for >> > some >> > repetitive tasks, used in multiple places / flows. Unfortunately, NiFi >> > templates are very limited in use for that purpose (you can only >> > import/export them but cannot sync changes in them across flows). >> > >> > Wanted to use Python, but realized that it was not a good idea because >> > NiFi >> > was using Jython that comes with a bunch of limitations. Groovy was >> > really >> > easy to learn for basic scripting. >> > >> > Boris >> > >> > On Tue, Jul 10, 2018 at 10:00 AM Ed B <bdes...@gmail.com> wrote: >> >> >> >> Hi James, >> >> >> >> I have implemented couple of custom processors, python- and >> >> groovy-based >> >> ISP and ES, and obviously implementations using miles-long flows. >> >> There are couple of aspects: development, deployment and maintenance. >> >> Our client, consider "code change" when you need to deploy new file on >> >> Linux box, so if you write anything in NIFI - it's not a code change, >> >> and >> >> doesn't require complex process of deployment activities, so we stick >> >> with >> >> ISP/ES unless it's not possible at all without Java (custom processor). >> >> Try to avoid Python, as ES is terribly slow with Python (actually, >> >> Jython). >> >> If your team won't be able to support your code, but they are OK with >> >> NIFI >> >> - avoid ISP/ES/custom processors unless absolutely required. Write long >> >> flows, structure them well in PGs, etc. Because any code requires >> >> maintenance. >> >> If you are writing more like a reusable component - then obviously >> >> single >> >> component (ISP/ES or custom processor) will make much more sense, as >> >> number >> >> of processors (duplicated by number of usages in different flows) will >> >> have >> >> an impact on memory and cpu. >> >> There are more, but these are my main considerations. >> >> Hope this helps. >> >> >> >> Regards, >> >> Ed. >> >> >> >> On Mon, Jul 9, 2018 at 3:30 PM James Srinivasan >> >> <james.sriniva...@gmail.com> wrote: >> >>> >> >>> Hi all, >> >>> >> >>> I was wondering if there is any general guidance about when to use >> >>> ExecuteScript and when to use a chain of processors? For example, in >> >>> one application I am downloading a HTML index file, extracting the >> >>> links corresponding to more index pages of data per year, fetching >> >>> those pages, extracting some more links per month and then downloading >> >>> the results. I'm currently doing this with a bunch of NiFi processors >> >>> (about 20 in total), whereas I could replace them all by a single >> >>> fairly simple Python or Groovy script called by ExecuteScript. >> >>> >> >>> In another application, I have written a custom processor but probably >> >>> could have written the same code in a script. >> >>> >> >>> Any guidance on how to choose between the three options would be much >> >>> appreciated (yay for choices!) >> >>> >> >>> Thanks very much, >> >>> >> >>> James
