softyoungha opened a new issue, #40850:
URL: https://github.com/apache/airflow/issues/40850
### Description
Hello,
I have recently been using operators based on
`_BasePythonVirtualenvOperator` quite extensively, and it would be beneficial
if environment variables could be injected at runtime.
I usually deploy on a k8s environment using Helm, and while it works fine if
I apply environment variables directly to the Airflow image, but this requires
Docker build & push registry & pod restart.
If it were possible to pass environment variables from the DAG file to the
operator, it would be convenient as it would allow execution based solely on
the DAG code synchronized with the Airflow UI and Git.
### Use case/motivation
I'm currently doing it this way.
```python
def my_python_func(env: str):
import os
os.environ["SOME_ENV"] = env
from some.package import module
...
external_python_task = ExternalPythonOperator(
task_id='my_external_python_task',
python='path/to/venv/bin/python',
python_callable=my_python_function,
op_args=["yeah"],
)
# or
@task.external_python(
task_id='my_external_python_task',
python='path/to/venv/bin/python',
)
def my_python_func(env: str):
import os
os.environ["SOME_ENV"] = env
from some.package import module
t = my_python_func("yeah")
```
It would be nice if we could use this in this way.
```python
def my_python_func(env: str):
from some.package import module
...
external_python_task = ExternalPythonOperator(
task_id='my_external_python_task',
python='path/to/venv/bin/python',
python_callable=my_python_function,
env_vars={"SOME_ENV": "yeah"}
)
@task.external_python(
task_id='my_external_python_task',
python='path/to/venv/bin/python',
env_vars={"SOME_ENV": "yeah"}
)
def my_python_func(env: str):
from some.package import module
t = my_python_func("yeah")
```
If we add this feature to `_BasePythonVirtualenvOperator`, it seems that it
could also be added to `PythonVirtualenvOperator` and
`BranchPythonVirtualenvOperator`.
### Related issues
_No response_
### Are you willing to submit a PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
