[GitHub] [airflow] habibdhif commented on issue #8605: Add Production-ready docker compose for the production image

[GitBox]

habibdhif commented on issue #8605:
URL: https://github.com/apache/airflow/issues/8605#issuecomment-623182960


   Here is another example of a Docker Compose that I've been working on. The 
Compose defines multiple services to run Airflow. 
   There is an init service which is an ephemeral container to initialize the 
database and creates a user if necessary. 
   The init service command tries to run `airflow list_users` and if it fails 
it initializes the database and creates a user. Different approaches were 
considered but this one is simple enough and only involves airflow commands (no 
database-specific commands).
   
   Extension fields are used for airflow environment variables to reduce code 
duplication.
   
   I added a Makefile along the docker-compose.yml in my 
[repo](https://github.com/habibdhif/docker-compose-airflow) so all you have to 
do to run the docker-compose is run `make run`.
   
   ```yaml
   version: "3.7"
   x-airflow-environment: &airflow-environment
     AIRFLOW__CORE__EXECUTOR: CeleryExecutor
     AIRFLOW__WEBSERVER__RBAC: "True"
     AIRFLOW__CORE__LOAD_EXAMPLES: "False"
     AIRFLOW__CELERY__BROKER_URL: "redis://:@redis:6379/0"
     AIRFLOW__CORE__SQL_ALCHEMY_CONN: 
postgresql+psycopg2://airflow:airflow@postgres:5432/airflow
   
   services:
     postgres:
       image: postgres:11.5
       environment:
         POSTGRES_USER: airflow
         POSTGRES_DB: airflow
         POSTGRES_PASSWORD: airflow
     redis:
       image: redis:5
       environment:
         REDIS_HOST: redis
         REDIS_PORT: 6379
       ports:
         - 6379:6379
     init:
       image: apache/airflow:1.10.10
       environment:
         <<: *airflow-environment
       depends_on:
         - redis
         - postgres
       volumes:
         - ./dags:/opt/airflow/dags
       entrypoint: /bin/bash
       command: >
         -c "airflow list_users || (airflow initdb
         && airflow create_user --role Admin --username airflow --password 
airflow -e airf...@airflow.com -f airflow -l airflow)"
       restart: on-failure
     webserver:
       image: apache/airflow:1.10.10
       ports:
         - 8080:8080
       environment:
         <<: *airflow-environment
       depends_on:
         - init
       volumes:
         - ./dags:/opt/airflow/dags
       command: "webserver"
       restart: always
     flower:
       image: apache/airflow:1.10.10
       ports:
         - 5555:5555
       environment:
         <<: *airflow-environment
       depends_on:
         - redis
       command: flower
       restart: always
     scheduler:
       image: apache/airflow:1.10.10
       environment:
         <<: *airflow-environment
       depends_on:
         - webserver
       volumes:
         - ./dags:/opt/airflow/dags
       command: scheduler
       restart: always
     worker:
       image: apache/airflow:1.10.10
       environment:
         <<: *airflow-environment
       depends_on:
         - scheduler
       volumes:
         - ./dags:/opt/airflow/dags
       command: worker
       restart: always
   ```


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org