Clustering not happening in Kubernetes cluster

2018-04-16 Thread Jonathan Kosgei
Hi, I'm trying to cluster nifi on kubernetes but haven't been able to get the pods to connect to each other. I have the following statefulset and services; https://gist.github.com/jonathan-kosgei/cdf7c9ec882948eac12beb6b28ffa748 I'm running nifi 1.6.0 on Kubernetes v1.8.8-gke.0. My

Re: Clustering not happening in Kubernetes cluster

2018-04-16 Thread Mark Payne
Jonathan, I've spent some time in the last few weeks playing around with getting some clusters running on Kubernetes as well. I know from the mailing lists that some others have been venturing into this also. I'm by no means a Kubernetes expert myself, but I've gotten it up & running on a

Re: Clustering not happening in Kubernetes cluster

2018-04-16 Thread Anil Rai
Hi Mark, We are also looking at spinning up nifi on k8s. If you could share more information on your architecture of Nifi on k8s, that would be very helpful. Also any best practice that you would suggest. I am interested in understanding the auto scaling of nifi on k8s. Before I deep dive, wanted

Re: Unit test for database Put processor

2018-04-16 Thread Mike Thomsen
Anthony, Sorry, forgot to answer your last question. The best thing you can do, especially since MarkLogic doesn't have a convenient Docker image published, is to do both unit and integration tests. A test starting or ending with Test is a unit test and one ending with IT is an integration test

Unit test for database Put processor

2018-04-16 Thread Anthony Roach
We are writing a PutMarkLogic processor to ingest flowfiles into a MarkLogic database. Looking at the PutMongo test, it appears that a running instance of MongoDB is expected. Attempting to run with one ends in failure. Can we make the same assumption for tests we include into the suite?

Re: Unit test for database Put processor

2018-04-16 Thread Mike Thomsen
The PutMongo tests fail because the Mongo processors haven't been fully migrated to the new method of declaring EL support. With that said, the Mongo tests aren't almost entirely integration tests, not unit tests because Mongo is kinda painful to mock. On Mon, Apr 16, 2018 at 5:50 PM Anthony