0x26res opened a new issue, #38280:
URL: https://github.com/apache/airflow/issues/38280
### Apache Airflow Provider(s)
amazon
### Versions of Apache Airflow Providers
apache-airflow-providers-amazon==8.7.1
### Apache Airflow version
2.7.2
### Operating System
ubuntu (or whatever MWAA provides)
### Deployment
Official Apache Airflow Helm Chart
### Deployment details
MWAA 2.7.2
### What happened
The EksPodOperator can submit a job, but the trigger fails later.
I think it's because the kube config is saved in a temp file that is gone by
the time the operator tries to poll the task progress.
```
File
"/usr/local/airflow/.local/lib/python3.11/site-packages/airflow/providers/cncf/kubernetes/triggers/pod.py",
line 145, in run
pod = await hook.get_pod(
^^^^^^^^^^^^^^^^^^^
File
"/usr/local/airflow/.local/lib/python3.11/site-packages/airflow/providers/cncf/kubernetes/hooks/kubernetes.py",
line 563, in get_pod
async with self.get_conn() as connection:
File "/usr/local/lib/python3.11/contextlib.py", line 204, in __aenter__
return await anext(self.gen)
^^^^^^^^^^^^^^^^^^^^^
File
"/usr/local/airflow/.local/lib/python3.11/site-packages/airflow/providers/cncf/kubernetes/hooks/kubernetes.py",
line 550, in get_conn
kube_client = await self._load_config() or async_client.ApiClient()
^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/usr/local/airflow/.local/lib/python3.11/site-packages/airflow/providers/cncf/kubernetes/hooks/kubernetes.py",
line 497, in _load_config
await async_config.load_kube_config(
File
"/usr/local/airflow/.local/lib/python3.11/site-packages/kubernetes_asyncio/config/kube_config.py",
line 566, in load_kube_config
loader = _get_kube_config_loader_for_yaml_file(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/usr/local/airflow/.local/lib/python3.11/site-packages/kubernetes_asyncio/config/kube_config.py",
line 530, in _get_kube_config_loader_for_yaml_file
return KubeConfigLoader(
^^^^^^^^^^^^^^^^^
File
"/usr/local/airflow/.local/lib/python3.11/site-packages/kubernetes_asyncio/config/kube_config.py",
line 148, in __init__
self.set_active_context(active_context)
File
"/usr/local/airflow/.local/lib/python3.11/site-packages/kubernetes_asyncio/config/kube_config.py",
line 159, in set_active_context
context_name = self._config['current-context']
~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
File
"/usr/local/airflow/.local/lib/python3.11/site-packages/kubernetes_asyncio/config/kube_config.py",
line 415, in __getitem__
```
### What you think should happen instead
The config file must be regenerated every time the trigger wakes up. The
content of the file should probably be saved in the operator to avoid fetching
it from the cluster every time.
### How to reproduce
Submit an EksPodOperator with `deferrable=True`
### Anything else
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
