wolfier opened a new issue, #55803:
URL: https://github.com/apache/airflow/issues/55803
### Apache Airflow Provider(s)
common-io
### Versions of Apache Airflow Providers
`apache-airflow-providers-common-io==1.6.2`
### Apache Airflow version
2.11.0
### Operating System
debian
### Deployment
Astronomer
### Deployment details
Fresh deployment running Runtime 13.2.0. This version uses Airflow 2.11.0
with `apache-airflow-providers-common-io==1.6.2`.
### What happened
Configured a custom XCom backend.
```
AIRFLOW__CORE__XCOM_BACKEND=airflow.providers.common.io.xcom.backend.XComObjectStorageBackend
```
The scheduler and triggerer are failing to star. The errors seem to be
originating from importing BaseOperator that is triggered by a custom plugin.
This happens to other plugins too.
```
[2025-09-17T18:32:13.715+0000] {plugins_manager.py:267} ERROR - Failed to
import plugin openlineage
Traceback (most recent call last):
File "/usr/local/lib/python3.12/site-packages/airflow/configuration.py",
line 1346, in getimport
return import_string(full_qualified_path)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/usr/local/lib/python3.12/site-packages/airflow/utils/module_loading.py", line
39, in import_string
module = import_module(module_path)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/importlib/__init__.py", line 90, in
import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 999, in exec_module
File "<frozen importlib._bootstrap>", line 488, in
_call_with_frames_removed
File
"/usr/local/lib/python3.12/site-packages/airflow/providers/common/io/xcom/backend.py",
line 30, in <module>
from airflow.providers.common.io.version_compat import AIRFLOW_V_3_0_PLUS
File
"/usr/local/lib/python3.12/site-packages/airflow/providers/common/io/version_compat.py",
line 40, in <module>
from airflow.models import BaseOperator
File "/usr/local/lib/python3.12/site-packages/airflow/models/__init__.py",
line 79, in __getattr__
val = import_string(f"{path}.{name}")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/usr/local/lib/python3.12/site-packages/airflow/utils/module_loading.py", line
39, in import_string
module = import_module(module_path)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/importlib/__init__.py", line 90, in
import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/usr/local/lib/python3.12/site-packages/airflow/models/baseoperator.py", line
83, in <module>
from airflow.models.mappedoperator import OperatorPartial,
validate_mapping_kwargs
File
"/usr/local/lib/python3.12/site-packages/airflow/models/mappedoperator.py",
line 54, in <module>
from airflow.triggers.base import StartTriggerArgs
File "/usr/local/lib/python3.12/site-packages/airflow/triggers/base.py",
line 27, in <module>
from airflow.models.taskinstance import SimpleTaskInstance
ImportError: cannot import name 'SimpleTaskInstance' from partially
initialized module 'airflow.models.taskinstance' (most likely due to a circular
import) (/usr/local/lib/python3.12/site-packages/airflow/models/taskinstance.py)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.12/site-packages/airflow/plugins_manager.py",
line 259, in load_entrypoint_plugins
plugin_class = entry_point.load()
^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/importlib/metadata/__init__.py", line 205,
in load
module = import_module(match.group('module'))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/importlib/__init__.py", line 90, in
import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 999, in exec_module
File "<frozen importlib._bootstrap>", line 488, in
_call_with_frames_removed
File
"/usr/local/lib/python3.12/site-packages/airflow/providers/openlineage/plugins/openlineage.py",
line 21, in <module>
from airflow.providers.openlineage.plugins.listener import
get_openlineage_listener
File
"/usr/local/lib/python3.12/site-packages/airflow/providers/openlineage/plugins/listener.py",
line 31, in <module>
from airflow.models import DagRun, TaskInstance
File "<frozen importlib._bootstrap>", line 1412, in _handle_fromlist
File "/usr/local/lib/python3.12/site-packages/airflow/models/__init__.py",
line 79, in __getattr__
val = import_string(f"{path}.{name}")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/usr/local/lib/python3.12/site-packages/airflow/utils/module_loading.py", line
39, in import_string
module = import_module(module_path)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/importlib/__init__.py", line 90, in
import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/airflow/models/dagrun.py",
line 60, in <module>
from airflow.models.taskinstance import TaskInstance as TI
File
"/usr/local/lib/python3.12/site-packages/airflow/models/taskinstance.py", line
104, in <module>
from airflow.models.xcom import LazyXComSelectSequence, XCom
File "/usr/local/lib/python3.12/site-packages/airflow/models/xcom.py",
line 815, in <module>
XCom = resolve_xcom_backend()
^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/airflow/models/xcom.py",
line 798, in resolve_xcom_backend
clazz = conf.getimport("core", "xcom_backend",
fallback=f"airflow.models.xcom.{BaseXCom.__name__}")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/airflow/configuration.py",
line 1349, in getimport
raise AirflowConfigException(
airflow.exceptions.AirflowConfigException: The object could not be loaded.
Please check "xcom_backend" key in "core" section. Current value:
"airflow.providers.common.io.xcom.backend.XComObjectStorageBackend".
```
### What you think should happen instead
The scheduler and triggerer should start up without issue with a custom XCom
backend.
### How to reproduce
1. Start a deployment with Runtime 13.2.0 (Airflow 2.11.0)
2. Configure the custom XCom backend to
`airflow.providers.common.io.xcom.backend.XComObjectStorageBackend`.
3. Start the deployment.
```
FROM quay.io/astronomer/astro-runtime:13.2.0
ENV
AIRFLOW__CORE__XCOM_BACKEND=airflow.providers.common.io.xcom.backend.XComObjectStorageBackend
```
### Anything else
This behaviour can be mitigated when `apache-airflow-providers-common-io`
version is pinned to `1.6.0`.
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
