Hi Joe The distributed cache service property of ListXXX doesn't seem to matter anymore. Which was quite a surprise to me. It just gets handled by the state manager: https://github.com/apache/nifi/blob/e68ff153e81ddb82d1136d44a96bdb7a70da86d1/nifi-nar-bundles/nifi-extension-utils/nifi-processor-utils/src/main/java/org/apache/nifi/processor/util/list/AbstractListProcessor.java#L264
You can test this via setting the Distributed Cache Service to a redis server and watch no connections ever being made, which was how I found out. Not sure if this is intended or not. On Wed, Sep 20, 2017 at 12:42 PM, Joe Witt <[email protected]> wrote: > Gino > > When the primary node shifts the new primary node should take over the > function of doing the listings for you. You'll want to be using the > distributed cache property of the ListFTP/etc.. processors to help > keep state in such cases. > > Thanks > > On Tue, Sep 19, 2017 at 4:50 PM, Gino Lisignoli <[email protected]> > wrote: > > Hi > > > > I can confirm that the cause of my problem is the primary node shifting > to a > > new node in my cluster. This causes all of the ListXXX processors to stop > > working (as they are set to only run on my primary node). I'm not sure > what > > is causing my primary node to shift, or why the processors are unable to > > recover when the node shifts. If I find any more info I'll raise a > ticket. > > > > On Thu, Sep 14, 2017 at 8:08 AM, Joe Witt <[email protected]> wrote: > >> > >> Thanks Gino. If you can confirm/replicate the behavior please do file a > >> jira. > >> > >> Thanks > >> > >> On Wed, Sep 13, 2017 at 4:06 PM, Gino Lisignoli <[email protected]> > >> wrote: > >> > In 1.4.0-SNAPSHOT, the ListFTP processor fails to add new files after > >> > several hours. > >> > > >> > At the moment I'm testing this on a cluster of 2. But I'm looking to > >> > replicate my issue on a single instance as well. > >> > > >> > I'm not sure if this is a cluster state problem, a processor state > >> > problem > >> > or the new ListXXX commits > >> > > >> > (https://github.com/apache/nifi/commit/e68ff153e81ddb82d1136d44a96bdb > 7a70da86d1 > >> > and > >> > > >> > https://github.com/apache/nifi/commit/28ee70222b892fb799f5f74a31a9de > 678d9fb629). > >> > > >> > I'll be looking at those commits first around the ListFTP process. > >> > Unless > >> > theres any advice can be offered. > >> > > >> > Should I submit this to Jira issues, or wait until I can > >> > confirm/replicate > >> > the behaviour? > > > > >
