ahipp13 opened a new issue, #68398:
URL: https://github.com/apache/airflow/issues/68398
### Under which category would you file this issue?
Airflow Core
### Apache Airflow version
3.2.2
### What happened and how to reproduce it?
I am running Airflow 3.2.2 on the 1.18.0 Helm Chart using the Kubernetes
executor. Airflow 3.2.1 worked just fine, but when I upgraded to Airflow 3.2.2
my schedulers started throwing the below error. I run 2 schedulers at once, so
I think what is happening is somehow there is a race condition where both of
them are trying to touch something the other has touched which is causing the
error. At first this error only happens to one scheduler, and then it affects
both of them and brings both of them down.
2026-06-11T17:13:14.598244Z [error ] Exception when executing
SchedulerJob._run_scheduler_loop
[airflow.jobs.scheduler_job_runner.SchedulerJobRunner]
loc=scheduler_job_runner.py:1551
Traceback (most recent call last):
File
"/home/airflow/.local/lib/python3.12/site-packages/airflow/jobs/scheduler_job_runner.py",
line 1546, in _execute
self._run_scheduler_loop()
File
"/home/airflow/.local/lib/python3.12/site-packages/airflow/jobs/scheduler_job_runner.py",
line 1612, in _run_scheduler_loop
self.adopt_or_reset_orphaned_tasks()
File
"/home/airflow/.local/lib/python3.12/site-packages/airflow/utils/session.py",
line 100, in wrapper
return func(*args, session=session, **kwargs) # type: ignore[arg-type]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/home/airflow/.local/lib/python3.12/site-packages/airflow/jobs/scheduler_job_runner.py",
line 2814, in adopt_or_reset_orphaned_tasks
for attempt in run_with_db_retries(logger=self.log):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/home/airflow/.local/lib/python3.12/site-packages/tenacity/__init__.py", line
438, in __iter__
do = self.iter(retry_state=retry_state)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/home/airflow/.local/lib/python3.12/site-packages/tenacity/__init__.py", line
371, in iter
result = action(retry_state)
^^^^^^^^^^^^^^^^^^^
File
"/home/airflow/.local/lib/python3.12/site-packages/tenacity/__init__.py", line
393, in <lambda>
self._add_action_func(lambda rs: rs.outcome.result())
^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/concurrent/futures/_base.py", line 449, in result
return self.__get_result()
^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/concurrent/futures/_base.py", line 401, in
__get_result
raise self._exception
File
"/home/airflow/.local/lib/python3.12/site-packages/airflow/jobs/scheduler_job_runner.py",
line 2864, in adopt_or_reset_orphaned_tasks
reset_tis_message.append(repr(ti))
^^^^^^^^
File
"/home/airflow/.local/lib/python3.12/site-packages/airflow/models/taskinstance.py",
line 1134, in __repr__
return prefix + f"[{self.state}] ti_id={self.id}>"
^^^^^^^^^^
File
"/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/orm/attributes.py",
line 569, in __get__
return self.impl.get(state, dict_) # type: ignore[no-any-return]
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/orm/attributes.py",
line 1096, in get
value = self._fire_loader_callables(state, key, passive)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/orm/attributes.py",
line 1129, in _fire_loader_callables
return callable_(state, passive)
^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/orm/strategies.py",
line 578, in __call__
return strategy._load_for_state(state, passive)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/orm/strategies.py",
line 536, in _load_for_state
raise orm_exc.DetachedInstanceError(
sqlalchemy.orm.exc.DetachedInstanceError: Parent instance <TaskInstance at
0x7fb1b521e390> is not bound to a Session; deferred load operation of attribute
'state' cannot proceed (Background on this error at:
https://sqlalche.me/e/20/bhk3)
2026-06-11T17:13:14.601149Z [info ] Shutting down Kubernetes executor
[airflow.providers.cncf.kubernetes.executors.kubernetes_executor.KubernetesExecutor]
loc=kubernetes_executor.py:867
### What you think should happen instead?
Schedulers should run as normal together
### Operating System
(airflow)cat /etc/os-release ID=chainguard NAME="Chainguard"
PRETTY_NAME="Chainguard" VERSION_ID="20230214"
HOME_URL="https://chainguard.dev/";
### Deployment
Official Apache Airflow Helm Chart
### Apache Airflow Provider(s)
_No response_
### Versions of Apache Airflow Providers
_No response_
### Official Helm Chart version
1.18.0
### Kubernetes Version
PS C:\Users\ahippe1> k version Client Version: v1.34.5-dispatcher Kustomize
Version: v5.7.1 Server Version: v1.34.3-gke.400
### Helm Chart configuration
My values file is huge, so if there are specific pieces you want to look at
please let me know and I will paste them in
### Docker Image customizations
_No response_
### Anything else?
When I run the command:
UPDATE task_instance
SET state = 'failed'
WHERE state IN ('running', 'queued')
in the Airflow database, it shuts down all running DAGs, but then it fixes
the problem and it does not happen...
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
