[
https://issues.apache.org/jira/browse/AIRFLOW-664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15713167#comment-15713167
]
Dennis O'Brien edited comment on AIRFLOW-664 at 12/1/16 9:45 PM:
-----------------------------------------------------------------
Screenshot of Airflow UI built from fdb7e949
!airflow_broken_scheduler_screenshot.png!
was (Author: dennisobrien):
Screenshot of Airflow UI built from fdb7e949
> airflow built from master unstable since change to parse DAG files in child
> process
> -----------------------------------------------------------------------------------
>
> Key: AIRFLOW-664
> URL: https://issues.apache.org/jira/browse/AIRFLOW-664
> Project: Apache Airflow
> Issue Type: Bug
> Components: scheduler
> Reporter: Dennis O'Brien
> Attachments: airflow_broken_scheduler_screenshot.png
>
>
> Prior to commit fdb7e949140b735b8554ae5b22ad752e86f6ebaf I was able to pip
> build, install, and run airflow from the github commit. E.g.,
> {code}
> pip install -e
> git://github.com/apache/incubator-airflow.git@835bcb6234956445d819ff39a5790f635c201c5d#egg=airflow[celery,crypto,hive,jdbc,ldap,password,postgres,s3,vertica]
> {code}
> Since this commit, airflow is not functional.
> * The on/off button to enable a DAG does not appear.
> * Next to each DAG an info icon appears with the hover text "This DAG seems
> to be existing only locally. The master scheduler doesn't seem to be aware of
> its existence."
> * DAGs cannot be run from the command line.
> I am able via the command line to list the dags and list the tasks for a dag,
> but trigger_dag does not actually cause any tasks to run (but no error
> messages either).
> {code}
> airflow@41e0e6920223:~$ airflow trigger_dag example_bash_operator
> [2016-12-01 19:24:21,928] {__init__.py:50} INFO - Using executor
> CeleryExecutor
> [2016-12-01 19:24:22,218] {models.py:162} INFO - Filling up the DagBag from
> /usr/local/airflow/dags
> [2016-12-01 19:24:22,341] {cli.py:180} INFO - Created <DagRun
> example_bash_operator @ 2016-12-01 19:24:22:
> manual__2016-12-01T19:24:22.295709, externally triggered: True>
> {code}
> This commit was #1636 "[AIRFLOW-160] Parse DAG files through child
> processes". I haven't identified what part of this commit is to blame. Or,
> what part of my configuration/setup is incompatible with this change.
> My setup:
> * Docker and docker-compose with...
> * Ubuntu 16.04
> * Python 3.5.2 (via Miniconda distribution)
> * Separate container instances for scheduler, webserver, flower, and worker.
> * Docker container running mysql 5.7
> * Docker container running redis 3.2
> * Using CeleryExecutor with redis broker
> * Building and installing from master using `pip install -e git://...`
> I have a github project that uses Docker and docker-compose to build and test
> these components.
> [https://github.com/dennisobrien/airflow-docker-compose-example] .
> * pip install line in
> [Dockerfile|https://github.com/dennisobrien/airflow-docker-compose-example/blob/master/Dockerfile#L58]
> * [airflow.cfg
> template|https://github.com/dennisobrien/airflow-docker-compose-example/blob/master/config/airflow.cfg.tpl]
> *
> [docker-entrypoint.sh|https://github.com/dennisobrien/airflow-docker-compose-example/blob/master/bin/docker-entrypoint.sh]
> defines how each service is run.
> *
> [docker-compose.yml|https://github.com/dennisobrien/airflow-docker-compose-example/blob/master/docker-compose.yml]
> orchestrates the components.
> Let me know if there are additional tests I can do to narrow this down. I
> suspect since others have not complained about this that it is probably
> something related to my setup.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
