[
https://issues.apache.org/jira/browse/AIRFLOW-1868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16830988#comment-16830988
]
jack commented on AIRFLOW-1868:
-------------------------------
Can you please check if this is still an issue in newer airflow version?
> Packaged Dags not added to dag table, unable to execute tasks
> -------------------------------------------------------------
>
> Key: AIRFLOW-1868
> URL: https://issues.apache.org/jira/browse/AIRFLOW-1868
> Project: Apache Airflow
> Issue Type: Bug
> Environment: airflow 1.8.2, celery, rabbitMQ, mySQL, aws
> Reporter: nathan warshauer
> Priority: Major
> Attachments: Screen Shot 2017-11-29 at 2.31.02 PM.png, Screen Shot
> 2017-11-29 at 4.40.39 PM.png, Screen Shot 2017-11-29 at 4.42.39 PM.png
>
>
> .zip files in the dag directory do not appear to be getting added to the dag
> table on the airflow database. When a .zip file is placed within the dags
> folder and it contains executable .py files, the dag_id should be added to
> the dag table and airflow should allow the dag to be unpaused and run through
> the web server.
> SELECT distinct dag.dag_id AS dag_dag_id FROM dag confirms the dag does not
> exist in the dags table but shows up on the UI with the warning message "This
> Dag seems to be existing only locally" however the dag exists in all 3 dag
> directories (master and two workers) and the airflow.cfg has donot_pickle =
> True
> When the dag is triggered manually via airflow trigger_dag <dag_id> the
> process goes to the web server and does not execute any tasks. When I go to
> the task and click start through the UI the task will execute successfully
> and shows the attached state upon completion. When I do not do this process
> the tasks will not enter the queue and the run sits idle as the 3rd attached
> image shows.
> Basically, the dag CAN run manually from the zip BUT the scheduler and
> underlying database tables appear to not be functioning correctly for
> packaged dags.
> Please let me know if I can provide any additional information regarding this
> issue, or if you all have any leads that I can check out for resolving this.
> dag = DAG('MY-DAG-NAME',
> default_args=default_args,
> schedule_interval='*/5 * * * *',
> max_active_runs=1,
> dagrun_timeout=timedelta(minutes=4, seconds=30))
> default_args = {
> 'depends_on_past': False,
> 'email': ['[email protected]'],
> 'email_on_failure': True,
> 'email_on_retry': False,
> 'owner': 'airflow',
> 'provide_context': True,
> 'retries': 0,
> 'retry_delay': timedelta(minutes=5),
> 'start_date': datetime(2017,11,28)
> }
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
