jens-scheffler-bosch commented on issue #34688:
URL: https://github.com/apache/airflow/issues/34688#issuecomment-1741814891
To make a regression I tested on latest main and the version tag 2.7.1 with
the following example DAG that installs the named venv inside a DAG preventing
that I need to re-build exactly your Docker environment. With this setup I can
not reproduce the error. Can you try on your side?
```
from __future__ import annotations
from datetime import datetime
from airflow.decorators import dag, task
from airflow.operators.bash import BashOperator
@dag(
schedule=None,
start_date=datetime(2021, 1, 1),
)
def example_break():
setup_external_python = BashOperator(
task_id="setup_external_python",
bash_command="python -m venv
/opt/airflow/python_venvs/operators_venv"
)
install_packages = BashOperator(
task_id="install_packages",
bash_command="/opt/airflow/python_venvs/operators_venv/bin/pip
install apache-airflow[postgres]==2.7.1"
)
@task.external_python(python="/opt/airflow/python_venvs/operators_venv/bin/python",
expect_airflow=True)
def this_should_break():
from airflow.providers.postgres.hooks.postgres import PostgresHook
print(f"I get some postgress hook object here: {PostgresHook()}")
return "yes"
cleanup_python = BashOperator(
task_id="cleanup_python",
bash_command="rm -Rvf /opt/airflow/python_venvs/operators_venv"
)
setup_external_python.as_setup() >> install_packages.as_setup() >>
this_should_break() >> cleanup_python.as_teardown()
example_break()
```
I assume you setup difference is somewhat related to the creation of your
venv. Can you test on your side and use the example to find a difference to
your prepared Docker setup?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
