RuizChristianJ opened a new issue, #35690:
URL: https://github.com/apache/airflow/issues/35690
### Apache Airflow version
2.7.3
### What happened
I am trying to create a kubernetes connection using a JSON for the
kubeconfig. I have tried both using an environment variable and an airflow
connection to no avail. I have tested this kubeconfig as a yaml; passing the
path to kube_config_path works as intended. But when converting that same yaml
to a json and passing it to kube_config breaks.
`[2023-11-16, 18:34:20 UTC] {base.py:73} INFO - Using connection ID 'deveks'
for task execution.
[2023-11-16, 18:34:20 UTC] {kubernetes_pod.py:667} ERROR - string indices
must be integers, not 'str'
Traceback (most recent call last):
File
"/usr/local/lib/python3.11/site-packages/airflow/providers/cncf/kubernetes/operators/kubernetes_pod.py",
line 440, in execute
self.pod_request_obj = self.build_pod_request_obj(context)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/usr/local/lib/python3.11/site-packages/airflow/providers/cncf/kubernetes/operators/kubernetes_pod.py",
line 630, in build_pod_request_obj
"airflow_kpo_in_cluster": str(self.hook.is_in_cluster),
^^^^^^^^^^^^^^^^^^^^^^^
File
"/usr/local/lib/python3.11/site-packages/airflow/providers/cncf/kubernetes/hooks/kubernetes.py",
line 238, in is_in_cluster
self.api_client # so we can determine if we are in_cluster or not
^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/functools.py", line 1001, in __get__
val = self.func(instance)
^^^^^^^^^^^^^^^^^^^
File
"/usr/local/lib/python3.11/site-packages/airflow/providers/cncf/kubernetes/hooks/kubernetes.py",
line 246, in api_client
return self.get_conn()
^^^^^^^^^^^^^^^
File
"/usr/local/lib/python3.11/site-packages/airflow/providers/cncf/kubernetes/hooks/kubernetes.py",
line 207, in get_conn
config.load_kube_config(
File
"/usr/local/lib/python3.11/site-packages/kubernetes/config/kube_config.py",
line 808, in load_kube_config
loader = _get_kube_config_loader(
^^^^^^^^^^^^^^^^^^^^^^^^
File
"/usr/local/lib/python3.11/site-packages/kubernetes/config/kube_config.py",
line 762, in _get_kube_config_loader
kcfg = KubeConfigMerger(filename)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/usr/local/lib/python3.11/site-packages/kubernetes/config/kube_config.py",
line 681, in __init__
self._load_config_from_file_path(paths)
File
"/usr/local/lib/python3.11/site-packages/kubernetes/config/kube_config.py",
line 706, in _load_config_from_file_path
self.load_config(path)
File
"/usr/local/lib/python3.11/site-packages/kubernetes/config/kube_config.py",
line 724, in load_config
self._merge(item, config.get(item, []) or [], path)
File
"/usr/local/lib/python3.11/site-packages/kubernetes/config/kube_config.py",
line 730, in _merge
if exists['name'] == new_item['name']:
~~~~~~^^^^^^^^
File
"/usr/local/lib/python3.11/site-packages/kubernetes/config/kube_config.py",
line 618, in __getitem__
v = self.safe_get(key)
^^^^^^^^^^^^^^^^^^
File
"/usr/local/lib/python3.11/site-packages/kubernetes/config/kube_config.py",
line 615, in safe_get
return self.value[key]
~~~~~~~~~~^^^^^
TypeError: string indices must be integers, not 'str'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File
"/usr/local/lib/python3.11/site-packages/airflow/providers/cncf/kubernetes/operators/kubernetes_pod.py",
line 483, in cleanup
for event in self.pod_manager.read_pod_events(pod).items:
^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/functools.py", line 1001, in __get__
val = self.func(instance)
^^^^^^^^^^^^^^^^^^^
File
"/usr/local/lib/python3.11/site-packages/airflow/providers/cncf/kubernetes/operators/kubernetes_pod.py",
line 370, in pod_manager
return PodManager(kube_client=self.client)
^^^^^^^^^^^
File "/usr/local/lib/python3.11/functools.py", line 1001, in __get__
val = self.func(instance)
^^^^^^^^^^^^^^^^^^^
File
"/usr/local/lib/python3.11/site-packages/airflow/providers/cncf/kubernetes/operators/kubernetes_pod.py",
line 388, in client
return self.hook.core_v1_client
^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/functools.py", line 1001, in __get__
val = self.func(instance)
^^^^^^^^^^^^^^^^^^^
File
"/usr/local/lib/python3.11/site-packages/airflow/providers/cncf/kubernetes/hooks/kubernetes.py",
line 250, in core_v1_client
return client.CoreV1Api(api_client=self.api_client)
^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/functools.py", line 1001, in __get__
val = self.func(instance)
^^^^^^^^^^^^^^^^^^^
File
"/usr/local/lib/python3.11/site-packages/airflow/providers/cncf/kubernetes/hooks/kubernetes.py",
line 246, in api_client
return self.get_conn()
^^^^^^^^^^^^^^^
File
"/usr/local/lib/python3.11/site-packages/airflow/providers/cncf/kubernetes/hooks/kubernetes.py",
line 207, in get_conn
config.load_kube_config(
File
"/usr/local/lib/python3.11/site-packages/kubernetes/config/kube_config.py",
line 808, in load_kube_config
loader = _get_kube_config_loader(
^^^^^^^^^^^^^^^^^^^^^^^^
File
"/usr/local/lib/python3.11/site-packages/kubernetes/config/kube_config.py",
line 762, in _get_kube_config_loader
kcfg = KubeConfigMerger(filename)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/usr/local/lib/python3.11/site-packages/kubernetes/config/kube_config.py",
line 681, in __init__
self._load_config_from_file_path(paths)
File
"/usr/local/lib/python3.11/site-packages/kubernetes/config/kube_config.py",
line 706, in _load_config_from_file_path
self.load_config(path)
File
"/usr/local/lib/python3.11/site-packages/kubernetes/config/kube_config.py",
line 724, in load_config
self._merge(item, config.get(item, []) or [], path)
File
"/usr/local/lib/python3.11/site-packages/kubernetes/config/kube_config.py",
line 730, in _merge
if exists['name'] == new_item['name']:
~~~~~~^^^^^^^^
File
"/usr/local/lib/python3.11/site-packages/kubernetes/config/kube_config.py",
line 618, in __getitem__
v = self.safe_get(key)
^^^^^^^^^^^^^^^^^^
File
"/usr/local/lib/python3.11/site-packages/kubernetes/config/kube_config.py",
line 615, in safe_get
return self.value[key]
~~~~~~~~~~^^^^^
TypeError: string indices must be integers, not 'str'`
### What you think should happen instead
It should be able to consume the provided json and establish a connection
like it does when using the kube_config yaml file.
### How to reproduce
### Operating System
Linux
### Versions of Apache Airflow Providers
I have tried 5.0.0 up to the latest release of 7.9.0 for
apache-airflow-providers-cncf-kubernetes
### Deployment
Astronomer
### Deployment details
_No response_
### Anything else
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
