Bryan, Thanks, I was thinking the same, may be get all the root level processors and filter them instead of going through each processor group(one call to each PG will be expensive).
-Karthik -----Original Message----- From: Bryan Bende [mailto:[email protected]] Sent: Tuesday, August 14, 2018 9:17 AM To: [email protected] Subject: Re: [EXT] Re: Get all Processors You could probably achieve the same thing but traversing the process groups and asking each one for its processors without the includeDescendantGroups=true. It would be more complex on the client side, but would avoid making one huge request. On Tue, Aug 14, 2018 at 11:07 AM, Karthik Kothareddy (karthikk) [CONT - Type 2] <[email protected]> wrote: > Hi Pierre, > > > > I tried this both on Standalone instance (1.7.1) and clustered > instance > (1.6.0) where the nifi.cluster.node.read.timeout is set to 60 secs. > > > > -Karthik > > > > > > From: Pierre Villard [mailto:[email protected]] > Sent: Tuesday, August 14, 2018 2:20 AM > To: [email protected] > Subject: Re: [EXT] Re: Get all Processors > > > > Hi Karthik, > > > > Are you running a cluster or standalone NiFi? What's your default read > timeout value in nifi.properties? I believe the default value is a bit > low when you start reaching thousands of processors. > > > > nifi.cluster.node.read.timeout=5 sec > > > > Pierre > > > > > > 2018-08-14 0:12 GMT+02:00 Karthik Kothareddy (karthikk) [CONT - Type > 2] > <[email protected]>: > > Joe, > > I tried this call on both 1.7.1 and 1.6.0 and still getting the > timeout exception. I know that this is a very expensive call and > requires lot of caching from serverside. I was looking for a way to > get all processors and the controller Services they refer (if any?). > Not sure how to get the information I need in one call. > > -Karthik > > > -----Original Message----- > From: Joe Witt [mailto:[email protected]] > Sent: Monday, August 13, 2018 2:07 PM > To: [email protected] > Subject: [EXT] Re: Get all Processors > > Karthik > > I believe that call is/was very expensive on the server side. You > might want to experiment with the latest release of NiFi against the > same flow configuration. From conversations I have had I feel like > this is an addressed issue though admittedly i'm not sure which JIRA > would address it if that is the case. > > Others might have better data offhand. > > Thanks > On Mon, Aug 13, 2018 at 3:08 PM Karthik Kothareddy (karthikk) [CONT - > Type 2] <[email protected]> wrote: >> >> All, >> >> >> >> I was trying to get all processors from “root” Process group with the >> following rest call - >> /nifi-api/process-groups/root/processors?includeDescendantGroups=true >> and this call keeps timing out with the below exception >> >> >> >> javax.ws.rs.ProcessingException: java.net.SocketTimeoutException: >> Read timed out >> >> >> >> We have around 2000 processors on that instance and if I change the >> process group from root to a lower level group with less number of >> processors, the call will return the ProcessorsEntity json. Any idea >> on why this is timing out where as more bulkier rest calls such as >> /flow/process-groups/root/status?recursive=true Will return results >> immediately ? >> >> >> >> >> >> -Karthik > >
