This should be correct behavior... the cache approach was used before the state manager got introduced, and the cache was left there to migrate old state, but we do need to pick a release and remove those lingering cache properties.
Do you have the cluster state manager configured in state managers xml? It should be using zoo keeper when in a cluster. On Wed, Sep 20, 2017 at 6:23 PM Gino Lisignoli <[email protected]> wrote: > Hi Joe > > The distributed cache service property of ListXXX doesn't seem to matter > anymore. Which was quite a surprise to me. It just gets handled by the > state manager: > https://github.com/apache/nifi/blob/e68ff153e81ddb82d1136d44a96bdb7a70da86d1/nifi-nar-bundles/nifi-extension-utils/nifi-processor-utils/src/main/java/org/apache/nifi/processor/util/list/AbstractListProcessor.java#L264 > > You can test this via setting the Distributed Cache Service to a redis > server and watch no connections ever being made, which was how I found out. > Not sure if this is intended or not. > > > On Wed, Sep 20, 2017 at 12:42 PM, Joe Witt <[email protected]> wrote: > >> Gino >> >> When the primary node shifts the new primary node should take over the >> function of doing the listings for you. You'll want to be using the >> distributed cache property of the ListFTP/etc.. processors to help >> keep state in such cases. >> >> Thanks >> >> On Tue, Sep 19, 2017 at 4:50 PM, Gino Lisignoli <[email protected]> >> wrote: >> > Hi >> > >> > I can confirm that the cause of my problem is the primary node shifting >> to a >> > new node in my cluster. This causes all of the ListXXX processors to >> stop >> > working (as they are set to only run on my primary node). I'm not sure >> what >> > is causing my primary node to shift, or why the processors are unable to >> > recover when the node shifts. If I find any more info I'll raise a >> ticket. >> > >> > On Thu, Sep 14, 2017 at 8:08 AM, Joe Witt <[email protected]> wrote: >> >> >> >> Thanks Gino. If you can confirm/replicate the behavior please do file >> a >> >> jira. >> >> >> >> Thanks >> >> >> >> On Wed, Sep 13, 2017 at 4:06 PM, Gino Lisignoli <[email protected]> >> >> wrote: >> >> > In 1.4.0-SNAPSHOT, the ListFTP processor fails to add new files after >> >> > several hours. >> >> > >> >> > At the moment I'm testing this on a cluster of 2. But I'm looking to >> >> > replicate my issue on a single instance as well. >> >> > >> >> > I'm not sure if this is a cluster state problem, a processor state >> >> > problem >> >> > or the new ListXXX commits >> >> > >> >> > ( >> https://github.com/apache/nifi/commit/e68ff153e81ddb82d1136d44a96bdb7a70da86d1 >> >> > and >> >> > >> >> > >> https://github.com/apache/nifi/commit/28ee70222b892fb799f5f74a31a9de678d9fb629 >> ). >> >> > >> >> > I'll be looking at those commits first around the ListFTP process. >> >> > Unless >> >> > theres any advice can be offered. >> >> > >> >> > Should I submit this to Jira issues, or wait until I can >> >> > confirm/replicate >> >> > the behaviour? >> > >> > >> > > -- Sent from Gmail Mobile
