[ 
https://issues.apache.org/jira/browse/AIRFLOW-664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dennis O'Brien updated AIRFLOW-664:
-----------------------------------
    Attachment: airflow_broken_scheduler_screenshot.png

Screenshot of Airflow UI built from fdb7e949


> airflow built from master unstable since change to parse DAG files in child 
> process
> -----------------------------------------------------------------------------------
>
>                 Key: AIRFLOW-664
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-664
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: scheduler
>            Reporter: Dennis O'Brien
>         Attachments: airflow_broken_scheduler_screenshot.png
>
>
> Prior to commit fdb7e949140b735b8554ae5b22ad752e86f6ebaf I was able to pip 
> build, install, and run airflow from the github commit.  E.g.,
> {code}
> pip install -e 
> git://github.com/apache/incubator-airflow.git@835bcb6234956445d819ff39a5790f635c201c5d#egg=airflow[celery,crypto,hive,jdbc,ldap,password,postgres,s3,vertica]
> {code}
> Since this commit, airflow is not functional.
> * The on/off button to enable a DAG does not appear.
> * Next to each DAG an info icon appears with the hover text "This DAG seems 
> to be existing only locally. The master scheduler doesn't seem to be aware of 
> its existence."
> * DAGs cannot be run from the command line.
> I am able via the command line to list the dags and list the tasks for a dag, 
> but trigger_dag does not actually cause any tasks to run (but no error 
> messages either).
> {code}
> airflow@41e0e6920223:~$ airflow trigger_dag example_bash_operator
> [2016-12-01 19:24:21,928] {__init__.py:50} INFO - Using executor 
> CeleryExecutor
> [2016-12-01 19:24:22,218] {models.py:162} INFO - Filling up the DagBag from 
> /usr/local/airflow/dags
> [2016-12-01 19:24:22,341] {cli.py:180} INFO - Created <DagRun 
> example_bash_operator @ 2016-12-01 19:24:22: 
> manual__2016-12-01T19:24:22.295709, externally triggered: True>
> {code}
> This commit was #1636 "[AIRFLOW-160] Parse DAG files through child 
> processes".  I haven't identified what part of this commit is to blame.  Or, 
> what part of my configuration/setup is incompatible with this change.
> My setup:
> * Docker and docker-compose with...
> * Ubuntu 16.04
> * Python 3.5.2 (via Miniconda distribution)
> * Separate container instances for scheduler, webserver, flower, and worker.
> * Docker container running mysql 5.7
> * Docker container running redis 3.2
> * Using CeleryExecutor with redis broker
> * Building and installing from master using `pip install -e git://...`
> I have a github project that uses Docker and docker-compose to build and test 
> these components.  
> [https://github.com/dennisobrien/airflow-docker-compose-example] .
> * pip install line in 
> [Dockerfile|https://github.com/dennisobrien/airflow-docker-compose-example/blob/master/Dockerfile#L58]
> * [airflow.cfg 
> template|https://github.com/dennisobrien/airflow-docker-compose-example/blob/master/config/airflow.cfg.tpl]
> * 
> [docker-entrypoint.sh|https://github.com/dennisobrien/airflow-docker-compose-example/blob/master/bin/docker-entrypoint.sh]
>  defines how each service is run.
> * 
> [docker-compose.yml|https://github.com/dennisobrien/airflow-docker-compose-example/blob/master/docker-compose.yml]
>  orchestrates the components.
> Let me know if there are additional tests I can do to narrow this down.  I 
> suspect since others have not complained about this that it is probably 
> something related to my setup.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Reply via email to