*TLDR: Cluster, multiple Coordinators (Setup script provided), Webinterface, Service (Foxx-App) Installation, what's the envisioned way to do so?*
Hello, first of all let me state right up front how awesome the cluster-feature in 3.x is, and that i really enjoyed the ease of setting it up in my docker environment. Job well done - kudos to the docker devs! I'm still left with one substential question, which i can't figure out my self. Im running docker 3.0.7 and use the follwoing scripts to bootstrap the setup on a distaster (HAV) and production (PROD) server instance: HAV: #!/bin/bash ## SETUP ## 3 Agencies: 2 HAV, 1 PROD ## 2 Primaries: 1 HAV, 1 PROD ## 2 Coordinators: 1 HAV, 1 PROD DOCKERIMAGE=arangodb:3.0.7 HAVIP="ip-of-hav-server" PRODIP="ip-of-prod-server" ROOTDIR="/home/user/dockertest" CUSTOMER="test" echo "INFO: Setting up ArangoDB Cluster: HAV-System ($HAVIP), corresponding PROD System is configured to be $PRODID" echo "INFO: ensure reachability of the configured ports!" echo "INFO: this script sets up the corresponding filestructure for the volumes" echo "INFO: preparing filestructure (if not existent) in $ROOTDIR" mkdir -p "${ROOTDIR}/arango-cluster-${CUSTOMER}/hav/agency1/data" mkdir -p "${ROOTDIR}/arango-cluster-${CUSTOMER}/hav/agency1/apps" mkdir -p "${ROOTDIR}/arango-cluster-${CUSTOMER}/hav/agency2/data" mkdir -p "${ROOTDIR}/arango-cluster-${CUSTOMER}/hav/agency2/apps" mkdir -p "${ROOTDIR}/arango-cluster-${CUSTOMER}/hav/db1/data" mkdir -p "${ROOTDIR}/arango-cluster-${CUSTOMER}/hav/db1/apps" mkdir -p "${ROOTDIR}/arango-cluster-${CUSTOMER}/hav/coord/data" mkdir -p "${ROOTDIR}/arango-cluster-${CUSTOMER}/hav/coord/apps" echo INFO: Starting first HAV agency on port 5000 ... docker run -d -p 5000:4001 -e ARANGO_NO_AUTH=1 --name=agency1hav \ -v "${ROOTDIR}/arango-cluster-${CUSTOMER}/hav/agency1/data":/var/lib/arangodb3 \ -v "${ROOTDIR}/arango-cluster-${CUSTOMER}/hav/agency1/apps":/var/lib/arangodb3-apps \ ${DOCKERIMAGE} \ --agency.id 0 \ --agency.size 3 \ --agency.wait-for-sync false \ --agency.supervision true \ --agency.supervision-frequency 5 \ --server.endpoint tcp://0.0.0.0:4001 \ --server.statistics false \ --server.threads 16 sleep 1 echo INFO: Starting second HAV agency on port 5001 ... docker run -d -p 5001:4001 -e ARANGO_NO_AUTH=1 --name=agency2hav \ -v "${ROOTDIR}/arango-cluster-${CUSTOMER}/hav/agency2/data":/var/lib/arangodb3 \ -v "${ROOTDIR}/arango-cluster-${CUSTOMER}/hav/agency2/apps":/var/lib/arangodb3-apps \ ${DOCKERIMAGE} \ --agency.id 1 \ --agency.size 3 \ --agency.wait-for-sync false \ --agency.supervision true \ --agency.supervision-frequency 5 \ --server.endpoint tcp://0.0.0.0:4001 \ --server.statistics false \ --server.threads 16 sleep 1 echo INFO: Starting primary HAV db server on port 5100... docker run -d -p 5100:4001 -e ARANGO_NO_AUTH=1 --name="dbserver1hav" \ -v "${ROOTDIR}/arango-cluster-${CUSTOMER}/hav/db1/data":/var/lib/arangodb3 \ -v "${ROOTDIR}/arango-cluster-${CUSTOMER}/hav/db1/apps":/var/lib/arangodb3-apps \ ${DOCKERIMAGE} \ --cluster.agency-endpoint tcp://${HAVIP}:5000 \ --cluster.agency-endpoint tcp://${HAVIP}:5001 \ --cluster.agency-endpoint tcp://${PRODIP}:5002 \ --cluster.my-address tcp://${HAVIP}:5100 \ --server.endpoint tcp://0.0.0.0:4001 \ --cluster.my-local-info HAVDB1 \ --cluster.my-role PRIMARY \ --log.level debug \ --server.statistics false echo INFO: Starting HAV coordinator on port 8530... docker run -d -e ARANGO_NO_AUTH=1 -p 8530:4001 --name="coord1hav" \ -v "${ROOTDIR}/arango-cluster-${CUSTOMER}/hav/coord/data":/var/lib/arangodb3 \ -v "${ROOTDIR}/arango-cluster-${CUSTOMER}/hav/coord/apps":/var/lib/arangodb3-apps \ ${DOCKERIMAGE} \ --cluster.agency-endpoint tcp://${HAVIP}:5000 \ --cluster.agency-endpoint tcp://${HAVIP}:5001 \ --cluster.agency-endpoint tcp://${PRODIP}:5002 \ --cluster.my-address tcp://${HAVIP}:8530 \ --server.endpoint tcp://0.0.0.0:4001 \ --cluster.my-local-info HAVCOORD1 \ --cluster.my-role COORDINATOR \ --log.level debug \ --server.statistics false PROD: #!/bin/bash ## SETUP ## 3 Agencies: 2 HAV, 1 PROD ## 2 Primaries: 1 HAV, 1 PROD ## 2 Coordinators: 1 HAV, 1 PROD DOCKERIMAGE=arangodb:3.0.7 HAVIP="ip-of-hav-server" PRODIP="ip-of-prod-server" ROOTDIR="/home/user/docker" CUSTOMER="test" echo "INFO: Setting up ArangoDB Cluster: PROD-System ($PRODID), corresponding HAV System is configured to be $HAVIP" echo "INFO: ensure reachability of the configured ports!" echo "INFO: this script setsup the corresponding filestructure for the volumes" echo "INFO: preparing filestructure (if not existent) in $ROOTDIR" mkdir -p "${ROOTDIR}/arango-cluster-${CUSTOMER}/prod/agency1/data" mkdir -p "${ROOTDIR}/arango-cluster-${CUSTOMER}/prod/agency1/apps" mkdir -p "${ROOTDIR}/arango-cluster-${CUSTOMER}/prod/db1/data" mkdir -p "${ROOTDIR}/arango-cluster-${CUSTOMER}/prod/db1/apps" mkdir -p "${ROOTDIR}/arango-cluster-${CUSTOMER}/prod/coord/data" mkdir -p "${ROOTDIR}/arango-cluster-${CUSTOMER}/prod/coord/apps" echo Starting first PROD agency on port 5002... docker run -d -p 5002:4001 -e ARANGO_NO_AUTH=1 --name=agency1prod \ -v "${ROOTDIR}/arango-cluster-${CUSTOMER}/prod/agency1/data":/var/lib/arangodb3 \ -v "${ROOTDIR}/arango-cluster-${CUSTOMER}/prod/agency1/apps":/var/lib/arangodb3-apps \ ${DOCKERIMAGE} \ --agency.id 2 \ --agency.size 3 \ --agency.wait-for-sync false \ --agency.supervision true \ --agency.supervision-frequency 5 \ --server.endpoint tcp://0.0.0.0:4001 \ --server.statistics false \ --agency.endpoint tcp://${HAVIP}:5000 \ --agency.endpoint tcp://${HAVIP}:5001 \ --agency.endpoint tcp://${PRODIP}:5002 \ --agency.notify true \ --server.threads 16 sleep 1 echo Starting primary PROD db server on port 5101... docker run -d -p 5101:4001 -e ARANGO_NO_AUTH=1 --name="dbserver1prod" \ -v "${ROOTDIR}/arango-cluster-${CUSTOMER}/prod/db1/data":/var/lib/arangodb3 \ -v "${ROOTDIR}/arango-cluster-${CUSTOMER}/prod/db1/apps":/var/lib/arangodb3-apps \ ${DOCKERIMAGE} \ --cluster.agency-endpoint tcp://${HAVIP}:5000 \ --cluster.agency-endpoint tcp://${HAVIP}:5001 \ --cluster.agency-endpoint tcp://${PRODIP}:5002 \ --cluster.my-address tcp://${PRODIP}:5101 \ --server.endpoint tcp://0.0.0.0:4001 \ --cluster.my-local-info PRODDB1 \ --cluster.my-role PRIMARY \ --log.level debug \ --server.statistics false echo Starting PROD coordinator on port 8531... docker run -d -e ARANGO_NO_AUTH=1 -p 8531:4001 --name="coord1prod" \ ${DOCKERIMAGE} \ --cluster.agency-endpoint tcp://${HAVIP}:5000 \ --cluster.agency-endpoint tcp://${HAVIP}:5001 \ --cluster.agency-endpoint tcp://${PRODIP}:5002 \ --cluster.my-address tcp://${PRODIP}:8531 \ --server.endpoint tcp://0.0.0.0:4001 \ --cluster.my-local-info PRODCOORD1 \ --cluster.my-role COORDINATOR \ --log.level debug \ --server.statistics false As you can see, i'm linking all the data and app dirs to my host. I have successfully added dbs and collections in this setup and tried some recover scenarios by stopping and restarting different parts of the cluster (read: primaries and coordinators). I stoped and removed docker container and run them again with the same host-volumes, successfully restoring the state of the cluster as well. So all should be good and im supposed to be happy right? Indeed i am - but: I struggle to install Foxx-Apps via the Webinterface reliably. Even when the cluster is up and running with all nodes beeing sound and healthy, i can't figure out how i'm supposed to install a Service troughout the whole cluster. I came across different scenarios so far: Once installing on one coordinato mirrored the service meta informations to the other coordinator. But on the second coordinator none of the sources was available and i could neither open the "API" Tab nor could i communicate with the service via the WEB-API. On the first coordinator everything worked perfectly. In another installation attempt all of a sudden it worked on both, but i since than could not replicate that success. So long story short - My Question: How is one supposed to install a Foxx-Service in a Cluster Setup with multiple Coordinators. (i'm under the Impression that i red something about it somewhere in the docs that stated you should install the service on every coordinator, but i can't find it anymore). Any hint/help would be greatly appreciated! Thanks in advance -- You received this message because you are subscribed to the Google Groups "ArangoDB" group. To unsubscribe from this group and stop receiving emails from it, send an email to arangodb+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.