Hi Joe, Thanks for the explanation, is there a Jira ticket for for a job management mechanism? Is this a priority for a coming release?
Is there a lag between events occurring and them becoming searchable, if so what settings help control this lag? Thanks, Eric On Tue, Nov 3, 2020 at 2:35 PM Joe Witt <joe.w...@gmail.com> wrote: > Eric, > > short version: Provenance queries can absolutely take away CPU time from > the flow. > > longer version: > They get the same priority as any other thread in nifi. Once prov queries > are being executed they use CPU. I would strongly advise against any > blending of the flow execution with provenance queries. What you're trying > to do though is a great idea and aligns to what Mark Payne has talked about > previous as a job management mechanism. This can/should be done without > provenance itself. > > > Thanks > > > > On Tue, Nov 3, 2020 at 3:31 PM Eric Secules <esecu...@gmail.com> wrote: > >> Hello everyone, >> >> I was wondering if it were possible for excessive use of the provenance >> api would cause flowfile processing to slow down and even come to a halt? >> My test setup queries the provenance API to see if all flowfiles that >> descended from a given input file have completed processing. This leads to >> a lot of requests to the provenance API and I fear it may be too much for >> the test VM to handle (4 vCPU and 16GB RAM). Do provenance query threads >> take precedence over scheduled task threads? >> >> Thanks, >> Eric >> >
