Jonathan, I've spent some time in the last few weeks playing around with getting some clusters running on Kubernetes as well. I know from the mailing lists that some others have been venturing into this also. I'm by no means a Kubernetes expert myself, but I've gotten it up & running on a few-node cluster in GCE, so I do have some familiarity.
One thing to note is that the log file that you shared appears to be from an instance that's been running for a while, not one that was newly started, so it's possible that there were some important details regarding why the clustering is not working as expected at the beginning of the logs that were missed. I also notice that you're setting the "nifi.web.http.host" and "nifi.cluster.node.address" properties to "nifi-test". I'm not sure that this will work. Perhaps it will, I've never tried that, but I don't know if the NiFi process will be able to listen on that hostname. What I ended up doing was to update my docker image locally to update the "hostname" variable to be the fully-qualified hostname: export hostname=$(hostname -f) And then set the http.host and node.address properties to that. If setting to nifi-test works, though, then that would be great, certainly a lot easier. But I suspect we'll need to take a closer look at the logs just after startup to see if they reveal anything interesting. Thanks -Mark On Apr 16, 2018, at 12:26 PM, Jonathan Kosgei <[email protected]<mailto:[email protected]>> wrote: Hi, I'm trying to cluster nifi on kubernetes but haven't been able to get the pods to connect to each other. I have the following statefulset and services; https://gist.github.com/jonathan-kosgei/cdf7c9ec882948eac12beb6b28ffa748 I'm running nifi 1.6.0 on Kubernetes v1.8.8-gke.0. My nifi-app.log: https://gist.github.com/jonathan-kosgei/527952976ec18cf3957cec4d1e186f68 I set every log option to DEBUG in logback.xml I've noticed a lot of restarts and several files in the logs folder nifi@nifi2-0:/opt/nifi/nifi-1.6.0/logs$ ls nifi-app.log nifi-app_2018-04-16_03.0.log nifi-app_2018-04-16_06.0.log nifi-app_2018-04-16_09.0.log nifi-app_2018-04-16_12.0.log nifi-app_2018-04-16_15.0.log nifi-app_2018-04-16_01.0.log nifi-app_2018-04-16_04.0.log nifi-app_2018-04-16_07.0.log nifi-app_2018-04-16_10.0.log nifi-app_2018-04-16_13.0.log nifi-bootstrap.log nifi-app_2018-04-16_02.0.log nifi-app_2018-04-16_05.0.log nifi-app_2018-04-16_08.0.log nifi-app_2018-04-16_11.0.log nifi-app_2018-04-16_14.0.log nifi-user.log Any ideas what I might be missing? The cluster configuration seemed very straightforward. Thanks!
