[
https://issues.apache.org/jira/browse/AIRFLOW-3415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16703594#comment-16703594
]
Yuri Bendana edited comment on AIRFLOW-3415 at 11/29/18 7:33 PM:
-----------------------------------------------------------------
[~ashb], I added it.
was (Author: ybendana):
[~ashb], I attached it.
> Imports become null when triggering dagruns in a loop
> -----------------------------------------------------
>
> Key: AIRFLOW-3415
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3415
> Project: Apache Airflow
> Issue Type: Bug
> Components: DagRun
> Affects Versions: 1.10.1
> Environment: CentOS 7
> Reporter: Yuri Bendana
> Priority: Minor
>
> When triggering dagruns in a loop, the imported references become null on the
> second iteration. Here is an example
> [gist|https://gist.github.com/ybendana/3bc0791fe00b099be04aca47a8d524c9]. For
> the purposes here, you can ignore the DagRunSensor task. On the first
> iteration the 'sleeper' dag gets triggered but on the second iteration I see a
> {noformat}
> TypeError: 'NoneType' object is not callable{noformat}
> To workaround this, I have to copy the import (in this case trigger_dag)
> inside the loop.
> Here is the stacktrace:
> {code:java}
> [2018-11-28 18:23:42,492] {{models.py:1789}} INFO - All retries failed;
> marking task as FAILED
> [2018-11-28 18:23:42,517] {{base_task_runner.py:101}} INFO - Job 1354170:
> Subtask t1 Traceback (most recent call last):
> [2018-11-28 18:23:42,518] {{base_task_runner.py:101}} INFO - Job 1354170:
> Subtask t1 File "/opt/venv/sgmo/bin/airflow", line 32, in <module>
> [2018-11-28 18:23:42,518] {{base_task_runner.py:101}} INFO - Job 1354170:
> Subtask t1 args.func(args)
> [2018-11-28 18:23:42,518] {{base_task_runner.py:101}} INFO - Job 1354170:
> Subtask t1 File
> "/opt/venv/sgmo/lib/python2.7/site-packages/airflow/utils/cli.py", line 74,
> in wrapper
> [2018-11-28 18:23:42,518] {{base_task_runner.py:101}} INFO - Job 1354170:
> Subtask t1 return f(*args, **kwargs)
> [2018-11-28 18:23:42,518] {{base_task_runner.py:101}} INFO - Job 1354170:
> Subtask t1 File
> "/opt/venv/sgmo/lib/python2.7/site-packages/airflow/bin/cli.py", line 490, in
> run
> [2018-11-28 18:23:42,519] {{base_task_runner.py:101}} INFO - Job 1354170:
> Subtask t1 _run(args, dag, ti)
> [2018-11-28 18:23:42,519] {{base_task_runner.py:101}} INFO - Job 1354170:
> Subtask t1 File
> "/opt/venv/sgmo/lib/python2.7/site-packages/airflow/bin/cli.py", line 406, in
> _run
> [2018-11-28 18:23:42,519] {{base_task_runner.py:101}} INFO - Job 1354170:
> Subtask t1 pool=args.pool,
> [2018-11-28 18:23:42,519] {{base_task_runner.py:101}} INFO - Job 1354170:
> Subtask t1 File
> "/opt/venv/sgmo/lib/python2.7/site-packages/airflow/utils/db.py", line 74, in
> wrapper
> [2018-11-28 18:23:42,519] {{base_task_runner.py:101}} INFO - Job 1354170:
> Subtask t1 return func(*args, **kwargs)
> [2018-11-28 18:23:42,520] {{base_task_runner.py:101}} INFO - Job 1354170:
> Subtask t1 File
> "/opt/venv/sgmo/lib/python2.7/site-packages/airflow/models.py", line 1659, in
> _run_raw_task
> [2018-11-28 18:23:42,520] {{base_task_runner.py:101}} INFO - Job 1354170:
> Subtask t1 result = task_copy.execute(context=context)
> [2018-11-28 18:23:42,520] {{base_task_runner.py:101}} INFO - Job 1354170:
> Subtask t1 File
> "/opt/venv/sgmo/lib/python2.7/site-packages/airflow/operators/python_operator.py",
> line 95, in execute
> [2018-11-28 18:23:42,520] {{base_task_runner.py:101}} INFO - Job 1354170:
> Subtask t1 return_value = self.execute_callable()
> [2018-11-28 18:23:42,520] {{base_task_runner.py:101}} INFO - Job 1354170:
> Subtask t1 File
> "/opt/venv/sgmo/lib/python2.7/site-packages/airflow/operators/python_operator.py",
> line 100, in execute_callable
> [2018-11-28 18:23:42,521] {{base_task_runner.py:101}} INFO - Job 1354170:
> Subtask t1 return self.python_callable(*self.op_args, **self.op_kwargs)
> [2018-11-28 18:23:42,521] {{base_task_runner.py:101}} INFO - Job 1354170:
> Subtask t1 File "/opt/airflow/dags/sleepersensor.py", line 11, in
> triggersleeper
> [2018-11-28 18:23:42,521] {{base_task_runner.py:101}} INFO - Job 1354170:
> Subtask t1 tdr = trigger_dag('sleeper')
> [2018-11-28 18:23:42,521] {{base_task_runner.py:101}} INFO - Job 1354170:
> Subtask t1 TypeError: 'NoneType' object is not callable
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
