o-nikolas commented on PR #29494:
URL: https://github.com/apache/airflow/pull/29494#issuecomment-1428888690
>The first traceback you
> I have not found yet which change triggered it (for sure that was not the
change by @o-nikolas #29257 as I reverted it and it did not help) but it seems
our main cli `airflow` fails with circular imports. Maybe others can find where
it is from but my changes fix it (@o-nikolas - this is another manifestation of
the circular "dependencies" we have - between config and settings):
>
> Example stack trace I got before that one in various stages of the fix:
>
> ```
> Traceback (most recent call last):
> File "/Users/jarek/.pyenv/versions/airflow-3.9/bin/airflow", line 5, in
<module>
> from airflow.__main__ import main
> File "/Users/jarek/IdeaProjects/airflow/airflow/__main__.py", line 27,
in <module>
> from airflow.cli import cli_parser
> File "/Users/jarek/IdeaProjects/airflow/airflow/cli/cli_parser.py", line
33, in <module>
> from airflow import settings
> File "/Users/jarek/IdeaProjects/airflow/airflow/settings.py", line 39,
in <module>
> from airflow.configuration import AIRFLOW_HOME, WEBSERVER_CONFIG, conf
# NOQA F401
> File "/Users/jarek/IdeaProjects/airflow/airflow/configuration.py", line
1794, in <module>
> conf.validate()
> File "/Users/jarek/IdeaProjects/airflow/airflow/configuration.py", line
343, in validate
> self._validate_config_dependencies()
> File "/Users/jarek/IdeaProjects/airflow/airflow/configuration.py", line
439, in _validate_config_dependencies
> executor, _ = ExecutorLoader.import_default_executor_cls()
> File
"/Users/jarek/IdeaProjects/airflow/airflow/executors/executor_loader.py", line
151, in import_default_executor_cls
> return cls.import_executor_cls(executor_name)
> File
"/Users/jarek/IdeaProjects/airflow/airflow/executors/executor_loader.py", line
126, in import_executor_cls
> return import_string(cls.executors[executor_name]),
ConnectorSource.CORE
> File
"/Users/jarek/IdeaProjects/airflow/airflow/utils/module_loading.py", line 36,
in import_string
> module = import_module(module_path)
> File
"/Users/jarek/.pyenv/versions/3.9.9/lib/python3.9/importlib/__init__.py", line
127, in import_module
> return _bootstrap._gcd_import(name[level:], package, level)
> File
"/Users/jarek/IdeaProjects/airflow/airflow/executors/sequential_executor.py",
line 30, in <module>
> from airflow.executors.base_executor import BaseExecutor, CommandType
> File
"/Users/jarek/IdeaProjects/airflow/airflow/executors/base_executor.py", line
34, in <module>
> from airflow.models.taskinstance import TaskInstance, TaskInstanceKey
> File "/Users/jarek/IdeaProjects/airflow/airflow/models/taskinstance.py",
line 72, in <module>
> from airflow.datasets.manager import dataset_manager
> File "/Users/jarek/IdeaProjects/airflow/airflow/datasets/manager.py",
line 27, in <module>
> from airflow.models.dataset import DatasetDagRunQueue, DatasetEvent,
DatasetModel
> File "/Users/jarek/IdeaProjects/airflow/airflow/models/dataset.py", line
40, in <module>
> from airflow.utils import timezone
> File "/Users/jarek/IdeaProjects/airflow/airflow/utils/timezone.py", line
27, in <module>
> from airflow.settings import TIMEZONE
> ImportError: cannot import name 'TIMEZONE' from partially initialized
module 'airflow.settings' (most likely due to a circular import)
(/Users/jarek/IdeaProjects/airflow/airflow/settings.py)
> ```
@potiuk I believe that a similar fix I applied in `airflow.__int__` (in
#29257) would also have fixed this case actually. Basically in `__main__.py`
if `airflow.configuration.conf` is imported **before**
`airflow.cli.cli_parser` (which imports `airflow.settings`) then the cycle
would be removed.
We get cycles like this when `airflow.settings` **is the first module to
import/load `airflow.configuration`** because the very first import of
`airflow.configuration` triggers the conf object to be created and validated,
but much of the conf validation code depends on `airflow.settings` downstream
so we can't have it happen inside `airflow.settings`.
So importing `airflow.configuration` early (thus initting/validating conf
**outside** the context of `airflow.settings`) would have fixed the issue in
`__main__.py` I think:
```diff
from __future__ import annotations
import os
import argcomplete
-from airflow.cli import cli_parser
from airflow.configuration import conf
+from airflow.cli import cli_parser
def main():
"""Main executable function."""
if conf.get("core", "security") == "kerberos":
os.environ["KRB5CCNAME"] = conf.get("kerberos", "ccache")
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
