fzkhouy opened a new issue, #27147:
URL: https://github.com/apache/airflow/issues/27147
### Apache Airflow version
2.4.1
### What happened
**When defining Kubernetes 'custom_resource_definition' of
'sparkApplication' as a path to a '.yaml' file:**
the dag fails with this Error:
```
[2022-10-19, 09:19:21 +01] {taskinstance.py:1851} ERROR - Task failed with
exception
Traceback (most recent call last):
File
"/home/airflow/.local/lib/python3.7/site-packages/airflow/providers/cncf/kubernetes/operators/spark_kubernetes.py",
line 76, in execute
namespace=self.namespace,
File
"/home/airflow/.local/lib/python3.7/site-packages/airflow/providers/cncf/kubernetes/hooks/kubernetes.py",
line 324, in create_custom_object
name=body_dict["metadata"]["name"],
TypeError: string indices must be integers
```
### What you think should happen instead
The normal behavior and referred to airflow documentation is to read the
application_file ".yaml" and get the `name=body["metadata"]["name"]` in my
case:
```
metadata:
name: spark
```
So, the dag can send a request to the Kubernetes cluster to execute the
spark application
### How to reproduce
- Set up the connection with kubernetes cluster
- Create a dag using SparkKubernetesOperator
- Provide your 'sparkApplication' as a path to a '.yaml' file
- Execute the dag and Hopefully, the same problem should occur
### Operating System
ubuntu 20.04
### Versions of Apache Airflow Providers
`apache-airflow-providers-cncf-kubernetes==4.4.0`
### Deployment
Docker-Compose
### Deployment details
- Docker-compose deployment using customized airflow image (just adding the
requirements already mentioned)
- The docker container is in a Linux server + we try to connect with a k8s
cluster ( so we can execute our spark applications
### Anything else
_No response_
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
