alex-astronomer opened a new issue #18592:
URL: https://github.com/apache/airflow/issues/18592
### Apache Airflow version
2.1.3
### Operating System
macOS Big Sur 11.3.1
### Versions of Apache Airflow Providers
n/a
### Deployment
Virtualenv installation
### Deployment details
completely stock, off the shelf, virtualenv installation with 2.1.3 and
Python 3.8
### What happened
When using the KubernetesPodOperator, env_vars cannot be used with a
template that is rendered as a native object. This is because of the function
`convert_env_vars` which is called in the `__init__()` function for
`kubernetes_pod.KubernetesPodOperator`. This is the contents of that function:
```
def convert_env_vars(env_vars) -> List[k8s.V1EnvVar]:
"""
Converts a dictionary into a list of env_vars
:param env_vars:
:return:
"""
if isinstance(env_vars, dict):
res = []
for k, v in env_vars.items():
res.append(k8s.V1EnvVar(name=k, value=v))
return res
elif isinstance(env_vars, list):
return env_vars
else:
raise AirflowException(f"Expected dict or list, got
{type(env_vars)}")
```
The (potential) template string or whatever object is passed to the
`env_vars` parameter in the constructor is called before that template has a
chance to evaluate. The evaluation happens somewhere between `__init__` and
`execute`. This means that any operation that we do on the template if that's
what we pass into the constructor is operated upon before it is templated. If
using the `render_template_as_native_obj=True` DAG parameter, this means that a
dictionary will be passed as a string. This function breaks if a template
string is passed in.
### What you expected to happen
I expect that the `convert_env_vars` function gets called after the template
string has a chance to be processed and rendered. This means that this should
be called after `__init__()` and before `execute`. Possibly at the beginning
of the execute function?
That way the template has a chance to render before we operate on it.
### How to reproduce
Create two tasks, one `PythonOperator` task and a `KubernetesPodOperator`
task. The python task returns a dictionary of env_vars as XCOM, and the K8s
Pod task uses that dictionary as its env_vars using an `ti.xcom_pull(...)` from
inside of a template string.
The k8s pod task will error out in parsing because the template string
hasn't been rendered at the time of the constructor call, and we are trying to
perform operations with it in the constructor.
### Anything else
Let me know if you have any questions about this issue or how to reproduce.
Thanks!
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
