csprl-nowigence opened a new issue #22368:
URL: https://github.com/apache/airflow/issues/22368
### Apache Airflow version
2.2.4 (latest released)
### What happened
Every time I try to import from a module in the dags/ folder, I get a DAGs
Import Error:
```
Broken DAG: [/root/airflow/dags/my_company/my_custom_dags/my_dag2.py]
Traceback (most recent call last):
File "<frozen importlib._bootstrap>", line 219, in
_call_with_frames_removed
File "/root/airflow/dags/my_company/my_custom_dags/my_dag2.py", line 8, in
<module>
from my_company.my_custom_dags.my_dag1 import my_function
ModuleNotFoundError: No module named 'my_company'
```
I cannot import from a module in the base dags folder, nor can I import from
packages laid out exactly like the documentation in Module Management >
[Typical structure of
packages](https://airflow.apache.org/docs/apache-airflow/stable/modules_management.html#typical-structure-of-packages).
### What you think should happen instead
Setting up a DAG package according to the documentation should allow
importing from modules in that package.
### How to reproduce
Installed airflow following Quick Start / [Running Airflow
Locally](https://airflow.apache.org/docs/apache-airflow/stable/start/local.html)
in a pyenv virtualenv on Ubuntu 20.04.
- Airflow version: 2.2.4
- Python version: 3.8.8
- Pyenv version: 2.2.3
```shell
# Install Airflow using the constraints file
AIRFLOW_VERSION=2.2.4
PYTHON_VERSION="$(python --version | cut -d " " -f 2 | cut -d "." -f 1-2)"
CONSTRAINT_URL="https://raw.githubusercontent.com/apache/airflow/constraints-${AIRFLOW_VERSION}/constraints-${PYTHON_VERSION}.txt";
pip install "apache-airflow==${AIRFLOW_VERSION}" --constraint
"${CONSTRAINT_URL}"
pip install apache-airflow-providers-mongo
# launch airflow
python -m airflow standalone
```
Created package structure based on Home / Modules Management > [Typical
structure of
packages](https://airflow.apache.org/docs/apache-airflow/stable/modules_management.html#typical-structure-of-packages).
```
# tree -a $( realpath dags )
/root/airflow/dags
├── .airflowignore
└── my_company
├── __init__.py
├── common_package
│ ├── __init__.py
│ ├── common_module.py
│ └── subpackage
│ ├── __init__.py
│ └── subpackaged_util_module.py
└── my_custom_dags
├── __init__.py
├── __pycache__
│ ├── my_dag1.cpython-38.pyc
│ └── my_dag2.cpython-38.pyc
├── base_dag.py
├── my_dag1.py
└── my_dag2.py
```
[my_dags_folder.zip](https://github.com/apache/airflow/files/8307146/my_dags_folder.zip)
Comment out all the imports from `my_company...` in `my_dag1.py` and
`my_dag2.py` and both dags load and run.
Uncomment any of those imports to receive a DAG Import Error.
### Operating System
Ubuntu 20.04.3 LTS
### Versions of Apache Airflow Providers
apache-airflow-providers-ftp==2.0.1
apache-airflow-providers-http==2.0.3
apache-airflow-providers-imap==2.2.0
apache-airflow-providers-mongo==2.3.0
apache-airflow-providers-sqlite==2.1.0
### Deployment
Virtualenv installation
### Deployment details
- Airflow version: 2.2.4
- Python version: 3.8.8
- Pyenv version: 2.2.3
- Ubuntu version: 20.04.3
### Anything else
```
#airflow info
Apache Airflow
version | 2.2.4
executor | SequentialExecutor
task_logging_handler | airflow.utils.log.file_task_handler.FileTaskHandler
sql_alchemy_conn | sqlite:////root/airflow/airflow.db
dags_folder | /root/airflow/dags
plugins_folder | /root/airflow/plugins
base_log_folder | /root/airflow/logs
remote_base_log_folder |
System info
OS | Linux
architecture | x86_64
uname | uname_result(system='Linux', node='...',
release='5.4.0-91-generic', version='#102-Ubuntu SMP Fri Nov 5 16:31:28 UTC
2021', machine='x86_64', processor='x86_64')
locale | ('en_US', 'UTF-8')
python_version | 3.8.8 (default, Apr 20 2021, 19:06:39) [GCC 9.3.0]
python_location | /root/.pyenv/versions/airflow/bin/python
Tools info
git | git version 2.25.1
ssh | OpenSSH_8.2p1 Ubuntu-4ubuntu0.3, OpenSSL 1.1.1f 31 Mar
2020
kubectl | NOT AVAILABLE
gcloud | NOT AVAILABLE
cloud_sql_proxy | NOT AVAILABLE
mysql | NOT AVAILABLE
sqlite3 | NOT AVAILABLE
psql | NOT AVAILABLE
Paths info
airflow_home | /root/airflow
system_path |
/root/.pyenv/versions/airflow/bin:/root/.pyenv/libexec:/root/.pyenv/plugins/python-build/bin:/root/.pyenv/plugins/pyenv-virtualenv/bin:/root/.pyenv/plugins/pyenv-update/bin:/root/.py
|
env/plugins/pyenv-installer/bin:/root/.pyenv/plugins/pyenv-doctor/bin:/root/.pyenv/plugins/pyenv-virtualenv/shims:/root/.pyenv/shims:/root/.pyenv/bin:/root/.pyenv/plugins/pyenv-virtu
|
alenv/shims:/root/.pyenv/shims:/root/.pyenv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
python_path |
/root/.pyenv/versions/3.8.8/envs/airflow/bin:/root/.pyenv/versions/3.8.8/lib/python38.zip:/root/.pyenv/versions/3.8.8/lib/python3.8:/root/.pyenv/versions/3.8.8/lib/python3.8/lib-dynl
|
oad:/root/.pyenv/versions/airflow/lib/python3.8/site-packages:/root/airflow/dags:/root/airflow/config:/root/airflow/plugins
airflow_on_path | True
Providers info
apache-airflow-providers-ftp | 2.0.1
apache-airflow-providers-http | 2.0.3
apache-airflow-providers-imap | 2.2.0
apache-airflow-providers-mongo | 2.3.0
apache-airflow-providers-sqlite | 2.1.0
```
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
