Yes I have three nodes on the NiFi cluster screen - two are CONNECTED and one
is CONNECTED,PRIMARY. All have up to date heartbeats. When I run
GenerateFlowFile with Timer Driven it executes on all three of my nodes.
However, I just switched the PRIMARY to another node via election and now it is
working. I moved it back to the one it was on previously and it is working
there too.
So it appears re-electing seemed to correct whatever state I was in that was
preventing "On Primary Node" functionality from working. Thanks for the help!
From: Mark Payne <marka...@hotmail.com>
To: users@nifi.apache.org; Tom Stewart <stewartthom...@yahoo.com>
Sent: Wednesday, April 13, 2016 1:16 PM
Subject: Re: 'On primary node' ListSFTP not working for new cluster
Tom,
It sounds like you do not have any node elected primary at all. If you click
the cluster icon in the top-right corner,it should show all of the nodes in
your cluster. Next to the nodes should be a ribbon that you can click to electa
new primary node... Though it should also show which node is currently the
primary. Can you check if it showsa primary node? And if so, can you verify
that the primary node is actually doing anything? I.e., if you start
GenerateFlowFileon all nodes, can you see that it is indeed running on the
primary node, in addition to the others?
Thanks-Mark
On Apr 13, 2016, at 2:01 PM, Tom Stewart <stewartthom...@yahoo.com> wrote:
On Primary Node - the Tasks/Time stays at zero for the 5 minute interval that
is displaying for me. When I flip it to Timer Driven it does increment as
expected. I have my Run Schedule at "60 sec".
The View State shows two keys (listing,timestamp/processed.timestamp), but the
Value is not changing. Both show Scope=Cluster. I think this is residual data
from when I had it set to "Timer Driven" and it actually processed some files.
I neglected to mention the version - this is 0.60.
I tried an even simpler flow with just GenerateFlowFile and LogAttribute and it
does the same for me.
From: Mark Payne <marka...@hotmail.com>
To: users@nifi.apache.org; Tom Stewart <stewartthom...@yahoo.com>
Sent: Wednesday, April 13, 2016 11:42 AM
Subject: Re: 'On primary node' ListSFTP not working for new cluster
Tom,
When you are running on Primary Node, do you see the "Tasks/Time" on the
processor showing that tasks are running?
If you right-click on the Processor and choose "View State", does it show
anything in the table?
Thanks-Mark
On Apr 13, 2016, at 11:59 AM, Tom Stewart <stewartthom...@yahoo.com> wrote:
I built a NiFi cluster and some test flows and things seem to be working fine.
My three nodes show in the cluster view and are all connected with one marked
PRIMARY. I cannot get 'On primary node' working with several processors I have
tried. My current one is a simple flow consisting of ListSFTP and LogAttribute.
If I set my ListSFTP to Timer Driven with Run schedule of 60 sec, it works
fine. However then it runs on all of my nodes. I changed it to On Primary Node
and while I see the log in the my primary node where it claims to start:
nifi-app.log:2016-04-13 10:53:55,598 DEBUG [StandardProcessScheduler
Thread-3-SendThread(los90hdf4.novalocal:2181)] org.apache.zookeeper.ClientCnxn
Reading reply sessionid:0x354103c2b860000, packet:: clientPath:null
serverPath:null finished:false header:: 2,4 replyHeader:: 2,55834574850,0
request:: '/nifi/components/4603bfe7-6d98-4ad4-99f2-2a740034ae03,F response::
#10000,s{51539607588,51539607597,1460559569213,1460560385643,8,0,0,0,5,0,51539607588}
nifi-app.log:2016-04-13 10:53:55,598 DEBUG [StandardProcessScheduler Thread-1]
o.a.nifi.processors.standard.ListSFTP
ListSFTP[id=4603bfe7-6d98-4ad4-99f2-2a740034ae03] Returning CLUSTER State:
StandardStateMap[version=8, values={}]
nifi-app.log:2016-04-13 10:53:55,599 INFO [StandardProcessScheduler Thread-4]
o.a.n.c.s.TimerDrivenSchedulingAgent Scheduled
ListSFTP[id=4603bfe7-6d98-4ad4-99f2-2a740034ae03] to run with 1 threads
However, I do not see any errors or accesses on my SFTP server where it is
actually ever attempting to connect. I can flip the processor back to Timer
Driven and my SFTP server starts seeing requests. But when I toggle back to On
Primary Node it doesn't appear to be executing. I think the other processor I
tried this with was GetHTTP with similar experience.
Curious if there are any debug steps or setting recommendations that are useful
to check if it appears that "On Primary Node" doesn't work for a cluster.
