khalidmammadov opened a new pull request #18359:
URL: https://github.com/apache/airflow/pull/18359
When failed dependencies checked as in below example, the command fails most
of the time and some time succeeds. This is due to race condition for the
session object.
It happens as dependency checker tries to access **ti.execution_date**
attribute which is **lazily** loaded and when load is required the session is
not available anymore as it was created temporarily for the duration _get_ti
call.
This PR fixes it by creating session beforehand and propagating to
downstream. This helps avoiding of creating multiple sessions and making sure
session is available when this lazy attribute is accessed.
I have also took opportunity to fix a typo for "**create_if_necssary**" ->
"**create_if_necessary**"
Thanks for reviewing!
root@c10237c871db:/opt/airflow/airflow/cli/commands# airflow tasks
failed-deps my_dag print_date manual__2021-09-11T00:00:00+00:00
[2021-09-19 13:16:44,512] {dagbag.py:492} INFO - Filling up the DagBag from
/files/dags
Traceback (most recent call last):
File "/usr/local/bin/airflow", line 33, in <module>
sys.exit(load_entry_point('apache-airflow', 'console_scripts',
'airflow')())
File "/opt/airflow/airflow/__main__.py", line 40, in main
args.func(args)
File "/opt/airflow/airflow/cli/cli_parser.py", line 48, in command
return func(*args, **kwargs)
File "/opt/airflow/airflow/utils/cli.py", line 92, in wrapper
return f(*args, **kwargs)
File "/opt/airflow/airflow/cli/commands/task_command.py", line 312, in
task_failed_deps
failed_deps = list(ti.get_failed_dep_statuses(dep_context=dep_context))
File "/opt/airflow/airflow/models/taskinstance.py", line 1042, in
get_failed_dep_statuses
for dep_status in dep.get_dep_statuses(self, session, dep_context):
File "/opt/airflow/airflow/ti_deps/deps/base_ti_dep.py", line 101, in
get_dep_statuses
yield from self._get_dep_statuses(ti, session, dep_context)
File
"/opt/airflow/airflow/ti_deps/deps/exec_date_after_start_date_dep.py", line 31,
in _get_dep_statuses
if ti.task.start_date and ti.execution_date < ti.task.start_date:
File
"/usr/local/lib/python3.6/site-packages/sqlalchemy/ext/associationproxy.py",
line 193, in __get__
return inst.get(obj)
File
"/usr/local/lib/python3.6/site-packages/sqlalchemy/ext/associationproxy.py",
line 546, in get
target = getattr(obj, self.target_collection)
File
"/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/attributes.py", line
294, in __get__
return self.impl.get(instance_state(instance), dict_)
File
"/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/attributes.py", line
730, in get
value = self.callable_(state, passive)
File
"/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/strategies.py", line
720, in _load_for_state
% (orm_util.state_str(state), self.key)
sqlalchemy.orm.exc.DetachedInstanceError: Parent instance <TaskInstance at
0x7fad6e3acc88> is not bound to a Session; lazy load operation of attribute
'dag_run' cannot proceed (Background on this error at:
http://sqlalche.me/e/13/bhk3)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
