V0lantis opened a new issue, #29701:
URL: https://github.com/apache/airflow/issues/29701
### Apache Airflow version
Other Airflow 2 version (please specify below)
### What happened
The scheduler falls in a **CrashLoopBackOff** when our
`zombie_detection_interval` conf param equals 30 seconds or higher. We get the
following Traceback:
```
[2023-02-22 16:12:49,081] {scheduler_job.py:762} ERROR - Exception when
executing SchedulerJob._run_scheduler_loop
Traceback (most recent call last):
File
"/home/airflow/.local/lib/python3.10/site-packages/airflow/jobs/scheduler_job.py",
line 745, in _execute
self._run_scheduler_loop()
File
"/home/airflow/.local/lib/python3.10/site-packages/airflow/jobs/scheduler_job.py",
line 877, in _run_scheduler_loop
next_event = timers.run(blocking=False)
File "/usr/local/lib/python3.10/sched.py", line 151, in run
action(*argument, **kwargs)
File
"/home/airflow/.local/lib/python3.10/site-packages/airflow/utils/event_scheduler.py",
line 37, in repeat
action(*args, **kwargs)
File
"/home/airflow/.local/lib/python3.10/site-packages/airflow/utils/session.py",
line 75, in wrapper
return func(*args, session=session, **kwargs)
File
"/home/airflow/.local/lib/python3.10/site-packages/airflow/jobs/scheduler_job.py",
line 1521, in _find_zombies
processor_subdir=ti.dag_model.processor_subdir,
File
"/home/airflow/.local/lib/python3.10/site-packages/sqlalchemy/orm/attributes.py",
line 482, in __get__
return self.impl.get(state, dict_)
File
"/home/airflow/.local/lib/python3.10/site-packages/sqlalchemy/orm/attributes.py",
line 942, in get
value = self._fire_loader_callables(state, key, passive)
File
"/home/airflow/.local/lib/python3.10/site-packages/sqlalchemy/orm/attributes.py",
line 978, in _fire_loader_callables
return self.callable_(state, passive)
File
"/home/airflow/.local/lib/python3.10/site-packages/sqlalchemy/orm/strategies.py",
line 863, in _load_for_state
raise orm_exc.DetachedInstanceError(
sqlalchemy.orm.exc.DetachedInstanceError: Parent instance <TaskInstance at
0x7f797b5ff040> is not bound to a Session; lazy load operation of attribute
'dag_model' cannot proceed (Background on this error at:
https://sqlalche.me/e/14/bhk3)
[2023-02-22 16:12:49,095] {scheduler_job.py:774} INFO - Exited execute loop
Traceback (most recent call last):
File "/home/airflow/.local/bin/airflow", line 8, in <module>
sys.exit(main())
File
"/home/airflow/.local/lib/python3.10/site-packages/airflow/__main__.py", line
39, in main
args.func(args)
File
"/home/airflow/.local/lib/python3.10/site-packages/airflow/cli/cli_parser.py",
line 52, in command
return func(*args, **kwargs)
File
"/home/airflow/.local/lib/python3.10/site-packages/airflow/utils/cli.py", line
103, in wrapper
return f(*args, **kwargs)
File
"/home/airflow/.local/lib/python3.10/site-packages/airflow/cli/commands/scheduler_command.py",
line 85, in scheduler
_run_scheduler_job(args=args)
File
"/home/airflow/.local/lib/python3.10/site-packages/airflow/cli/commands/scheduler_command.py",
line 50, in _run_scheduler_job
job.run()
File
"/home/airflow/.local/lib/python3.10/site-packages/airflow/jobs/base_job.py",
line 247, in run
self._execute()
File
"/home/airflow/.local/lib/python3.10/site-packages/airflow/jobs/scheduler_job.py",
line 745, in _execute
self._run_scheduler_loop()
File
"/home/airflow/.local/lib/python3.10/site-packages/airflow/jobs/scheduler_job.py",
line 877, in _run_scheduler_loop
next_event = timers.run(blocking=False)
File "/usr/local/lib/python3.10/sched.py", line 151, in run
action(*argument, **kwargs)
File
"/home/airflow/.local/lib/python3.10/site-packages/airflow/utils/event_scheduler.py",
line 37, in repeat
action(*args, **kwargs)
File
"/home/airflow/.local/lib/python3.10/site-packages/airflow/utils/session.py",
line 75, in wrapper
return func(*args, session=session, **kwargs)
File
"/home/airflow/.local/lib/python3.10/site-packages/airflow/jobs/scheduler_job.py",
line 1521, in _find_zombies
processor_subdir=ti.dag_model.processor_subdir,
File
"/home/airflow/.local/lib/python3.10/site-packages/sqlalchemy/orm/attributes.py",
line 482, in __get__
return self.impl.get(state, dict_)
File
"/home/airflow/.local/lib/python3.10/site-packages/sqlalchemy/orm/attributes.py",
line 942, in get
value = self._fire_loader_callables(state, key, passive)
File
"/home/airflow/.local/lib/python3.10/site-packages/sqlalchemy/orm/attributes.py",
line 978, in _fire_loader_callables
return self.callable_(state, passive)
File
"/home/airflow/.local/lib/python3.10/site-packages/sqlalchemy/orm/strategies.py",
line 863, in _load_for_state
raise orm_exc.DetachedInstanceError(
sqlalchemy.orm.exc.DetachedInstanceError: Parent instance <TaskInstance at
0x7f797b5ff040> is not bound to a Session; lazy load operation of attribute
'dag_model' cannot proceed (Background on this error at:
https://sqlalche.me/e/14/bhk3)
```
When the setting is around 15 seconds, it works just fine.
Airflow version **2.4.3**
### What you think should happen instead
There should not be any side effect when prolonging this setting
### How to reproduce
Increasing `zombie_detection_interval` should do the trick (I hope)
### Operating System
```PRETTY_NAME="Debian GNU/Linux 11 (bullseye)" NAME="Debian GNU/Linux"
VERSION_ID="11" VERSION="11 (bullseye)" VERSION_CODENAME=bullseye ID=debian
HOME_URL="https://www.debian.org/" SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"```
### Versions of Apache Airflow Providers
```
apache-airflow==2.4.3
apache-airflow-providers-amazon==4.1.0
apache-airflow-providers-celery==3.0.0
apache-airflow-providers-cncf-kubernetes==4.1.0
apache-airflow-providers-common-sql==1.2.0
apache-airflow-providers-datadog==3.0.0
apache-airflow-providers-docker==3.0.0
apache-airflow-providers-ftp==3.0.0
apache-airflow-providers-github==2.0.0
apache-airflow-providers-google==8.4.0
apache-airflow-providers-hashicorp==3.2.0
apache-airflow-providers-http==3.0.0
apache-airflow-providers-imap==3.0.0
apache-airflow-providers-jira==3.0.0
apache-airflow-providers-mysql==3.0.0
apache-airflow-providers-postgres==5.1.0
apache-airflow-providers-redis==3.1.0
apache-airflow-providers-salesforce==5.3.0
apache-airflow-providers-sftp==4.0.0
apache-airflow-providers-slack==5.0.0
apache-airflow-providers-snowflake==3.0.0
apache-airflow-providers-sqlite==3.2.1
apache-airflow-providers-ssh==3.0.0
apache-airflow-providers-tableau==3.0.0
apache-airflow-providers-zendesk==4.0.0
google-cloud-orchestration-airflow==1.4.1
lbc-data-airflow-okta-integration==0.10.0
```
### Deployment
Official Apache Airflow Helm Chart
### Deployment details
_No response_
### Anything else
_No response_
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]