Dennis O'Brien created AIRFLOW-664:

             Summary: airflow built from master unstable since change to parse 
DAG files in child process
                 Key: AIRFLOW-664
             Project: Apache Airflow
          Issue Type: Bug
          Components: scheduler
            Reporter: Dennis O'Brien

Prior to commit fdb7e949140b735b8554ae5b22ad752e86f6ebaf I was able to pip 
build, install, and run airflow from the github commit.  E.g.,

pip install -e 

Since this commit, airflow is not functional.
* The on/off button to enable a button does not appear.
* Next to each DAG an info icon appears with the hover text "This DAG seems to 
be existing only locally. The master scheduler doesn't seem to be aware of its 
* DAGs cannot be run from the command line.

I am able via the command line to list the dags and list the tasks for a dag, 
but trigger_dag does not actually cause any tasks to run (but no error messages 
airflow@41e0e6920223:~$ airflow trigger_dag example_bash_operator
[2016-12-01 19:24:21,928] {} INFO - Using executor CeleryExecutor
[2016-12-01 19:24:22,218] {} INFO - Filling up the DagBag from 
[2016-12-01 19:24:22,341] {} INFO - Created <DagRun 
example_bash_operator @ 2016-12-01 19:24:22: 
manual__2016-12-01T19:24:22.295709, externally triggered: True>

This commit was #1636 "[AIRFLOW-160] Parse DAG files through child processes".  
I haven't identified what part of this commit is to blame.  Or, what part of my 
configuration/setup is incompatible with this change.

My setup:
* Docker and docker-compose with...
* Ubuntu 16.04
* Python 3.5.2 (via Miniconda distribution)
* Separate container instances for scheduler, webserver, flower, and worker.
* Docker container running mysql 5.7
* Docker container running redis 3.2
* Using CeleryExecutor with redis broker
* Building and installing from master using `pip install -e git://...`

I have a github project that uses Docker and docker-compose to build and test 
these components.  
[] .
* pip install line in 
* [airflow.cfg 
 defines how each service is run.
 orchestrates the components.

Let me know if there are additional tests I can do to narrow this down.  I 
suspect since others have not complained about this that it is probably 
something related to my setup.

This message was sent by Atlassian JIRA

Reply via email to