Re: NIFI single node in cluster mode
ion for a specific period of time, which if I > recall is > > fairly high (I think 5 minutes?). If you lower this it'll still wait > for an > > election but will complete faster (we do 30 seconds, but you could do > > lower). There's a property controlling this. > > > > On Fri, Oct 12, 2018 at 5:41 PM Milan Das wrote: > > > > > Hello Nifi team, > > > > > > Is it possible to run a single NIFI node in cluster mode ? I have > this > > > requirement because we will add other nodes soon down line. > > > > > > I tried that by setting “nifi.cluster.is.node” but and zookeeper > > setting. > > > But seems it waits ever for election. > > > > > > > > > > > > Appreciate your thoughts. > > > > > > > > > > > > Thanks, > > > > > > Milan Das > > > > > > > > > -- > Sent from Gmail Mobile > > > >
Re: NIFI single node in cluster mode
Milan, If you haven't already done so, please take a look at the NiFi Admin Guide's sections "Securing Zookeeper" [1] and "Kerberizing NiFi’s ZooKeeper Client" [2], which should help you configure NiFi to use a kerberized ZooKeeper. [1] https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#securing_zookeeper [2] https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#zk_kerberos_client On Sat, Oct 13, 2018 at 9:38 AM Milan Das wrote: > Problem is I am using Kerbrized zookeeper and it is failing to create nifi > basepath. Even if TGT is getting created Authentication is failing. > > > 2018-10-13 13:33:53,573 INFO [Thread-12] org.apache.zookeeper.Login TGT > refresh thread started. > 2018-10-13 13:33:53,576 INFO [Thread-12] org.apache.zookeeper.Login TGT > valid starting at:Sat Oct 13 13:33:53 UTC 2018 > 2018-10-13 13:33:53,576 INFO [Thread-12] org.apache.zookeeper.Login TGT > expires: Sun Oct 14 13:33:53 UTC 2018 > 2018-10-13 13:33:53,577 INFO [Thread-12] org.apache.zookeeper.Login TGT > refresh sleeping until: Sun Oct 14 09:38:53 UTC 2018 > 2018-10-13 13:33:53,577 INFO > [main-SendThread(ip-172-30-1-132.ec2.internal:2181)] > o.a.zookeeper.client.ZooKeeperSaslClient Client will use GSSAPI as SASL > mechanism. > 2018-10-13 13:33:53,606 INFO [main-EventThread] > o.a.c.f.state.ConnectionStateManager State change: CONNECTED > 2018-10-13 13:33:53,616 ERROR > [main-SendThread(ip-172-30-1-132.ec2.internal:2181)] > o.a.zookeeper.client.ZooKeeperSaslClient SASL authentication failed using > login context 'Client'. > 2018-10-13 13:33:53,723 WARN [main] > o.a.n.c.l.e.CuratorLeaderElectionManager Unable to determine the Elected > Leader for role 'Cluster Coordinator' due to > org.apache.zookeeper.KeeperException$AuthFailedException: KeeperErrorCode = > AuthFailed for /nifi/leaders/Cluster Coordinator; assuming no leader has > been elected > 2018-10-13 13:33:53,724 INFO [Curator-Framework-0] > o.a.c.f.imps.CuratorFrameworkImpl backgroundOperationsLoop exiting > 2018-10-13 13:33:53,726 INFO [main] > o.apache.nifi.controller.FlowController It appears that no Cluster > Coordinator has been Elected yet. Registering for Cluster Coordinator Role. > > > Thanks, > Milan Das > > > On 10/12/18, 6:26 PM, "Bryan Bende" wrote: > > There is also another property for the # of candidates to wait for when > voting, if it sees the # of candidates first it will short circuit the > time > period. So setting the candidates to 1 for a single node cluster should > start immediately. > > On Fri, Oct 12, 2018 at 5:59 PM Jon Logan wrote: > > > It waits for election for a specific period of time, which if I > recall is > > fairly high (I think 5 minutes?). If you lower this it'll still wait > for an > > election but will complete faster (we do 30 seconds, but you could do > > lower). There's a property controlling this. > > > > On Fri, Oct 12, 2018 at 5:41 PM Milan Das wrote: > > > > > Hello Nifi team, > > > > > > Is it possible to run a single NIFI node in cluster mode ? I have > this > > > requirement because we will add other nodes soon down line. > > > > > > I tried that by setting “nifi.cluster.is.node” but and zookeeper > > setting. > > > But seems it waits ever for election. > > > > > > > > > > > > Appreciate your thoughts. > > > > > > > > > > > > Thanks, > > > > > > Milan Das > > > > > > > > > -- > Sent from Gmail Mobile > > > >
Re: NIFI single node in cluster mode
Problem is I am using Kerbrized zookeeper and it is failing to create nifi
basepath. Even if TGT is getting created Authentication is failing.
2018-10-13 13:33:53,573 INFO [Thread-12] org.apache.zookeeper.Login TGT refresh
thread started.
2018-10-13 13:33:53,576 INFO [Thread-12] org.apache.zookeeper.Login TGT valid
starting at:Sat Oct 13 13:33:53 UTC 2018
2018-10-13 13:33:53,576 INFO [Thread-12] org.apache.zookeeper.Login TGT
expires: Sun Oct 14 13:33:53 UTC 2018
2018-10-13 13:33:53,577 INFO [Thread-12] org.apache.zookeeper.Login TGT refresh
sleeping until: Sun Oct 14 09:38:53 UTC 2018
2018-10-13 13:33:53,577 INFO
[main-SendThread(ip-172-30-1-132.ec2.internal:2181)]
o.a.zookeeper.client.ZooKeeperSaslClient Client will use GSSAPI as SASL
mechanism.
2018-10-13 13:33:53,606 INFO [main-EventThread]
o.a.c.f.state.ConnectionStateManager State change: CONNECTED
2018-10-13 13:33:53,616 ERROR
[main-SendThread(ip-172-30-1-132.ec2.internal:2181)]
o.a.zookeeper.client.ZooKeeperSaslClient SASL authentication failed using login
context 'Client'.
2018-10-13 13:33:53,723 WARN [main] o.a.n.c.l.e.CuratorLeaderElectionManager
Unable to determine the Elected Leader for role 'Cluster Coordinator' due to
org.apache.zookeeper.KeeperException$AuthFailedException: KeeperErrorCode =
AuthFailed for /nifi/leaders/Cluster Coordinator; assuming no leader has been
elected
2018-10-13 13:33:53,724 INFO [Curator-Framework-0]
o.a.c.f.imps.CuratorFrameworkImpl backgroundOperationsLoop exiting
2018-10-13 13:33:53,726 INFO [main] o.apache.nifi.controller.FlowController It
appears that no Cluster Coordinator has been Elected yet. Registering for
Cluster Coordinator Role.
Thanks,
Milan Das
On 10/12/18, 6:26 PM, "Bryan Bende" wrote:
There is also another property for the # of candidates to wait for when
voting, if it sees the # of candidates first it will short circuit the time
period. So setting the candidates to 1 for a single node cluster should
start immediately.
On Fri, Oct 12, 2018 at 5:59 PM Jon Logan wrote:
> It waits for election for a specific period of time, which if I recall is
> fairly high (I think 5 minutes?). If you lower this it'll still wait for
an
> election but will complete faster (we do 30 seconds, but you could do
> lower). There's a property controlling this.
>
> On Fri, Oct 12, 2018 at 5:41 PM Milan Das wrote:
>
> > Hello Nifi team,
> >
> > Is it possible to run a single NIFI node in cluster mode ? I have this
> > requirement because we will add other nodes soon down line.
> >
> > I tried that by setting “nifi.cluster.is.node” but and zookeeper
> setting.
> > But seems it waits ever for election.
> >
> >
> >
> > Appreciate your thoughts.
> >
> >
> >
> > Thanks,
> >
> > Milan Das
> >
> >
>
--
Sent from Gmail Mobile
Re: NIFI single node in cluster mode
It waits for election for a specific period of time, which if I recall is fairly high (I think 5 minutes?). If you lower this it'll still wait for an election but will complete faster (we do 30 seconds, but you could do lower). There's a property controlling this. On Fri, Oct 12, 2018 at 5:41 PM Milan Das wrote: > Hello Nifi team, > > Is it possible to run a single NIFI node in cluster mode ? I have this > requirement because we will add other nodes soon down line. > > I tried that by setting “nifi.cluster.is.node” but and zookeeper setting. > But seems it waits ever for election. > > > > Appreciate your thoughts. > > > > Thanks, > > Milan Das > >
