Re: NiFi ExecuteScript vs multiple processors vs custom processor

2018-07-10 Thread Boris Tyukin
good to know, thanks Bryan! On Tue, Jul 10, 2018 at 11:37 AM Bryan Bende wrote: > You can do nested versioning since the beginning in 0.1.0. > > A common scenario might be to have several teams build different > versioned flows, and then someone who is in charge of deploying them > will create

Re: NiFi ExecuteScript vs multiple processors vs custom processor

2018-07-10 Thread Kevin Doran
There's a lot of great discussion on this thread. I’ll add that if you intend to use NiFi Registry with NiFi (which has lot of advantages, some of which have already been discussed), you’ll want to consider what is going to work best with NiFi Registry and your flow deployment/promotion

Re: NiFi ExecuteScript vs multiple processors vs custom processor

2018-07-10 Thread Bryan Bende
You can do nested versioning since the beginning in 0.1.0. A common scenario might be to have several teams build different versioned flows, and then someone who is in charge of deploying them will create another version PG that combines the nested versioned process groups of each of these teams.

Re: NiFi ExecuteScript vs multiple processors vs custom processor

2018-07-10 Thread Boris Tyukin
thanks Bryan. I saw your blog post on that. I think with registry 0.1 it was not possible to version nested PGs within parent PGs so I could not have "templatized" PG which has it is own version and use that PG with other versioned PGs. Has it changed with registry 0.2 now out? On Tue, Jul 10,

Re: NiFi ExecuteScript vs multiple processors vs custom processor

2018-07-10 Thread Mike Thomsen
As a rule of thumb, I would strongly suggest using the scripting capabilities wherever something "feels" like a script and you neither need the best possible performance nor to bring in new Java dependencies. If it is more of a core component of your business logic, needs to be thoroughly tested,

Re: NiFi ExecuteScript vs multiple processors vs custom processor

2018-07-10 Thread Bryan Bende
Boris, Regarding templates being limited... templates were really made as a way to share example flows, or help with debugging if you need to send someone your flow. Unfortunately they turned into a deployment mechanism since there wasn't a better solution at the time. Using NiFi Registry should

Re: NiFi ExecuteScript vs multiple processors vs custom processor

2018-07-10 Thread Boris Tyukin
I like Ed's recommendations and doing something similar. I use ISPs for some repetitive tasks, used in multiple places / flows. Unfortunately, NiFi templates are very limited in use for that purpose (you can only import/export them but cannot sync changes in them across flows). Wanted to use

Re: NiFi ExecuteScript vs multiple processors vs custom processor

2018-07-10 Thread Ed B
Hi James, I have implemented couple of custom processors, python- and groovy-based ISP and ES, and obviously implementations using miles-long flows. There are couple of aspects: development, deployment and maintenance. Our client, consider "code change" when you need to deploy new file on Linux

NiFi ExecuteScript vs multiple processors vs custom processor

2018-07-09 Thread James Srinivasan
Hi all, I was wondering if there is any general guidance about when to use ExecuteScript and when to use a chain of processors? For example, in one application I am downloading a HTML index file, extracting the links corresponding to more index pages of data per year, fetching those pages,