The resolution was to manually paginate the query statement with the appropriate postgres syntax and set the fetchSize to 250 instead of 0, which is unlimited.
On Mon, Sep 11, 2023 at 8:54 AM <e-soci...@gmx.fr> wrote: > > Hello Mike, > > Could you please give me the details about the resolution ? > Have you change something in the processor or just changing the sql > command ? > > Regards > > *Envoyé:* samedi 2 septembre 2023 à 00:00 > *De:* "Mike Thomsen" <mikerthom...@gmail.com> > *À:* users@nifi.apache.org > *Objet:* NiFi hanging during large sql query > I have a three node cluster with an executesqlrecord processor with > primary execution only. The sql it runs is a straight forward select on a > table with about 44m records. If I leave it running, after about 10 min the > node becomes unresponsive and leaves the cluster. The query runs just fine > in jetbrains data grip on that postgresql server, so I don’t think it’s > anything weird with the db or query. Any ideas about what could be causing > this? Even with a high limit like 5m records the query doesn’t lock up the > NiFi node. > > Sent from my iPhone > > >
