Hi Stephane, I've been using Docker compose to form a NiFi cluster for testing purpose. It's not on dockerhub, but I've shared it as a Github project. [1]
My knowledge of Docker is fairly limited, but hope this helps. 1) Although I haven't played with those yet, docker-compose has resource allocation parameters, such as cpu_quota or mem_limit [2]. I assume these can be helpful to allocate resources in a container pool using Swarm together. >From NiFi's perspective, conf/bootstrap.conf has JVM options. It's possible to tweak the bootstrap.conf when NiFi starts on a container, please see the example shell script I used to replace nifi.properties hostname setting [3] 2) docker-compose has 'scale' command. With that, you can spawn new NiFi nodes within the same network and get it automatically added to the cluster, e.g. $ docker-compose scale nifi-nodes=2 I have configured a Zk node, and a NiFi seed node to expose ports, and nifi-nodes as services in docker-compose.yml [4] 3) Docker container can map local file system to container volume. If you'd like to preserve NiFi data dirs such as content-repository across container's life cycle, then I think you can use this feature. [5] [1] https://github.com/ijokarumawak/docker-compose-nifi-cluster [2] https://docs.docker.com/compose/compose-file/#/cpushares-cpuquota-cpuset-domainname-hostname-ipc-macaddress-memlimit-memswaplimit-privileged-readonly-restart-shmsize-stdinopen-tty-user-workingdir [3] https://github.com/ijokarumawak/docker-compose-nifi-cluster/blob/master/nifi-node/start-nifi.sh [4] https://github.com/ijokarumawak/docker-compose-nifi-cluster/blob/master/docker-compose.yml [5] https://docs.docker.com/engine/tutorials/dockervolumes/ Thanks, Koji On Tue, Oct 11, 2016 at 3:03 PM, Stéphane Maarek <[email protected]> wrote: > Hi, > > Has anyone deployed NiFi using docker? I see many images on dockerhub, but I > have a few questions regarding usage: > > 1) How do you handle overall configuration (java memory options, etc) ? > 2) If you add a docker process, does the spun up NiFi get automatically > added to your cluster? > 3) How do you handle persistence of data ? > > Thanks! > Stephane
