Nifi Clustering not working

2018-04-17 Thread Jonathan Kosgei
Thank you for the tips.



I've tried setting the fqdn as the hosts and made sure each was pingeable from 
all the other nodes with no luck.



In particular I see this error a lot;





Unable to locate group with id 'd4779129-0162-1000-982f-9436d01eb801'.



I'm wondering if I might have a port closed that should be open, is there any 
list of all official ports? Looking in my logs the bootstrap port seems to be 
different each time and I wonder if maybe they need to be accessible for each 
pod?



Here's a trimmed down version of my logs (not in DEBUG mode);



nifi-app.log

https://gist.github.com/jonathan-kosgei/81936b7e59a563f55111bfe5be06e8bc



nifi-user.log

https://gist.github.com/jonathan-kosgei/19ce9929a764a40ac74a84fdcac2b3e2



nifi-bootrap.log

https://gist.github.com/jonathan-kosgei/e7396f558cbcd9a20cf6c0dca72c2728



My Statefulset and Services

https://gist.github.com/jonathan-kosgei/49a628edab109ba39da8598caf0e4f1e



Thanks!



Clustering not happening in Kubernetes cluster

2018-04-16 Thread Jonathan Kosgei
Hi,



I'm trying to cluster nifi on kubernetes but haven't been able to get the pods 
to connect to each other.



I have the following statefulset and services;

https://gist.github.com/jonathan-kosgei/cdf7c9ec882948eac12beb6b28ffa748



I'm running nifi 1.6.0 on Kubernetes v1.8.8-gke.0.



My nifi-app.log:

https://gist.github.com/jonathan-kosgei/527952976ec18cf3957cec4d1e186f68



I set every log option to DEBUG in logback.xml



I've noticed a lot of restarts and several files in the logs folder



nifi@nifi2-0:/opt/nifi/nifi-1.6.0/logs$ ls

nifi-app.log  nifi-app_2018-04-16_03.0.log  
nifi-app_2018-04-16_06.0.log  nifi-app_2018-04-16_09.0.log  
nifi-app_2018-04-16_12.0.log  nifi-app_2018-04-16_15.0.log

nifi-app_2018-04-16_01.0.log  nifi-app_2018-04-16_04.0.log  
nifi-app_2018-04-16_07.0.log  nifi-app_2018-04-16_10.0.log  
nifi-app_2018-04-16_13.0.log  nifi-bootstrap.log

nifi-app_2018-04-16_02.0.log  nifi-app_2018-04-16_05.0.log  
nifi-app_2018-04-16_08.0.log  nifi-app_2018-04-16_11.0.log  
nifi-app_2018-04-16_14.0.log  nifi-user.log



Any ideas what I might be missing?



The cluster configuration seemed very straightforward.



Thanks!