Hi David, > But I want to use supervisor. And below is the terrible error I get
We've been using supervisord successfully to start and manage Airflow processes for a while now. Below is our /etc/supervisord.conf file. The only quirk with our setup is that we use Python 3.5 to run Airflow, but Supervisor is only officially supported on Python 2 so we use pyenv to install 2.7.11 just to run Supervisor. We modify /usr/local/bin/supervisord and supervisorctl files to start with: #!/root/.pyenv/versions/2.7.11/bin/python Hope this helps, -Joe ; Configuration for Airflow webserver and scheduler in Supervisor [program:webserver] command=/usr/local/bin/airflow webserver stopsignal=QUIT stopasgroup=true user=root stdout_logfile=/var/log/airflow/webserver-stdout.log stderr_logfile=/var/log/airflow/webserver-stderr.log environment=HOME="/root",AIRFLOW_HOME="/usr/local/airflow",TMPDIR="/tmp/airflow" pidfile = /usr/local/airflow/airflow-webserver.pid [program:scheduler] command=/usr/local/bin/airflow scheduler -n 5 stopsignal=QUIT stopasgroup=true killasgroup=true user=root stdout_logfile=/var/log/airflow/scheduler-stdout.log stderr_logfile=/var/log/airflow/scheduler-stderr.log environment=HOME="/root",AIRFLOW_HOME="/usr/local/airflow",TMPDIR="/tmp/airflow" autorestart=true [program:worker] command=/usr/local/bin/airflow worker stopsignal=QUIT stopasgroup=true killasgroup=true user=root stdout_logfile=/var/log/airflow/worker-stdout.log stderr_logfile=/var/log/airflow/worker-stderr.log environment=HOME="/root",AIRFLOW_HOME="/usr/local/airflow",TMPDIR="/tmp/airflow" autorestart=true [program:flower] command=/usr/local/bin/celery flower --broker=redis://localhost:6379/0 --basic_auth=airflow:%(ENV_FLOWER_PASSWORD)s %(ENV_FLOWER_CERTFILE_OPTION)s %(ENV_FLOWER_KEYFILE_OPTION)s stopsignal=QUIT stopasgroup=true killasgroup=true user=root stdout_logfile=/var/log/airflow/flower-stdout.log stderr_logfile=/var/log/airflow/flower-stderr.log environment=HOME="/root",AIRFLOW_HOME="/usr/local/airflow",TMPDIR="/tmp/airflow" autorestart=true [supervisord] logfile = /tmp/supervisord.log logfile_maxbytes = 50MB logfile_backups=10 loglevel = info pidfile = /tmp/supervisord.pid [unix_http_server] file=/tmp/supervisor.sock [supervisorctl] serverurl = unix:///tmp/supervisor.sock prompt = airflow [rpcinterface:supervisor] supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface
