Mohit, I’m a little confused by some of the details you report. Initially you said there were ~100 flows with 25 processors each (2500); now there are 25,000. If you examine the logs, you note that “reaching the election process” takes the majority of the time — what messages are being printed in the log before the election process starts? Have you taken thread dumps at regular intervals during this time to see where the time is being spent?
This could be caused by the cost of fingerprinting such a large flow during cluster startup. Are there any custom processors in your NiFi instance? Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com He/Him PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Aug 16, 2020, at 10:28 PM, Mohit Jain <mo...@open-insights.com> wrote: > > Hi Pierre, > > No, the election process takes only a minute or two max, reaching the > election process takes a lot of time. > There are around 25k stopped/invalid components in the UI, I marked all of > them as disabled hoping it would resolve the issue but no improvement in the > startup time. > > On a similar note, NiFi Rest api response is also quite slow when there are a > lot of components. Would it improve by disabling the components which are not > in use? Or is there something else that could have been causing the issue? > > Thanks, > Mohit > > On Thu, Aug 13, 2020 at 9:49 PM Pierre Villard <pierre.villard...@gmail.com > <mailto:pierre.villard...@gmail.com>> wrote: > Hi, > > I'm surprised this is something you observe in the election process part. > I've constantly seen quick startup times even with thousands of components in > the flow. > I'd look into the logs and maybe turn on some debug logs to find out what's > going on. > > Pierre > > Le jeu. 13 août 2020 à 16:33, Joe Witt <joe.w...@gmail.com > <mailto:joe.w...@gmail.com>> a écrit : > Mohit, > > You almost certainly want to take that same flow and setup a cluster on a > more recent version of NiFi to compare startup times. For flows with > thousands of components there are important improvements which have occurred > in the past year and a half. > > Startup time, user perceived behavior in the UI on continuous operations, > etc.. have been improved. Further you can now hot load new versions of nars > which should reduce the need to restart. > > We also will have 1.12 out hopefully within days so that could be interesting > for you as well. > > Thanks > > On Thu, Aug 13, 2020 at 7:18 AM Mohit Jain <mo...@open-insights.com > <mailto:mo...@open-insights.com>> wrote: > Hi Team, > > I am using a single node NiFi 1.9.0 cluster. It takes more than 30 minutes to > start each time it is restarted. There are more than 100 flows on the NiFi UI > with an average of 25 processors per flow. It takes around 25-30 minutes to > reach the cluster election process after which it gets started in a minute. > > Is this an expected behaviour that startup time is directly proportional to > the number of processors in the Canvas? Or is there a way to reduce the NiFi > startup time? > > Any leads would be appreciated. > > Thanks, > Mohit
