+1 to both of those points: 1) Avoid validating that which it doesn't help (disabled and running) 2) Avoid using web/synchronous threading for any user code
On Tue, Nov 8, 2016 at 7:43 AM, Matt Gilman <[email protected]> wrote: > I also agreed these changes make sense. In addition, another approach we > could consider that has been discussed in the past [1] is to perform > component validation asynchronously. This presents its own challenges but > would also be helpful. We should try to avoid calling into user code in any > web thread. > > Matt > > [1] https://issues.apache.org/jira/browse/NIFI-950 > > On Mon, Nov 7, 2016 at 6:15 PM, Matt Burgess <[email protected]> wrote: > >> Agreed. Also we validate processors on a timer-based strategy in >> FlowController (looks like for snapshotting) and in the web server >> (via ControllerFacade), those seem to happen 6-7 times on that >> interval (which is like 15-20 seconds). Also we validate all >> processors on any change to the canvas (such as moving a processor). >> Besides Mike's suggestion, perhaps we should look at a purely >> event-driven strategy for validating processors if possible? >> >> Regards, >> Matt >> >> On Mon, Nov 7, 2016 at 6:06 PM, Joe Witt <[email protected]> wrote: >> > Makes good sense to me. >> > >> > On Nov 7, 2016 5:39 PM, "Michael Moser" <[email protected]> wrote: >> > >> >> All, >> >> >> >> I would like to propose a fundamental change to processor validation >> based >> >> on observations in https://issues.apache.org/jira/browse/NIFI-2996. I >> >> would >> >> like to validate processors only when they are in the STOPPED state. >> >> >> >> The properties on a processor in the RUNNING state should always be >> valid, >> >> else you should not have been able to start the processor. A processor >> in >> >> the DISABLED statue doesn't show validation results, so it seems a >> waste to >> >> validate its properties. >> >> >> >> The reason I'm proposing this change is because the NiFi UI slows down >> as >> >> you add more processors and controller services to the graph. Beyond >> common >> >> sense expectations that this would be true, it appears that processor >> >> validation is a significant part of the 'cost' on the server when >> >> responding to REST API requests. Some details from my testing are in >> the >> >> JIRA ticket. >> >> >> >> Thoughts? >> >> >> >> Thanks, >> >> -- Mike >> >> >>
