codeprasan opened a new issue, #33451: URL: https://github.com/apache/airflow/issues/33451
### Apache Airflow version 2.6.3 ### What happened While Spark Operator YAML using SparkKubernetesOperator below error is shown. Our Team noticed this error had earlier resolved as part of https://github.com/apache/airflow/issues/27147 however not addressed as part of Airflow2.6.3. Could you please make sure to address this issue in 2.6.3 and upcoming versions as well? File "/opt/app-root/lib64/python3.8/site-packages/airflow/providers/cncf/kubernetes/operators/spark_kubernetes.py", line 85, in execute name = body["metadata"]["name"] TypeError: string indices must be integers ### What you think should happen instead The SparkKubernetesOperator should able to call the Spark Operator YAML file without any issues ### How to reproduce - Set up the connection with kubernetes cluster - Create a dag using SparkKubernetesOperator - Provide your 'sparkApplication' as a path to a '.yaml' file - Execute the dag and Hopefully, the same problem should occur Below code will help to reproduce this error. from airflow.providers.cncf.kubernetes.hooks.kubernetes import KubernetesHook, _load_body_to_dict body = _load_body_to_dict('test.yaml') print(body) name = body["metadata"]["name"] print(name) ### Operating System FROM registry.access.redhat.com/ubi8/python-38 ### Versions of Apache Airflow Providers apache-airflow-providers-cncf-kubernetes==7.1.0 ### Deployment Other Docker-based deployment ### Deployment details We are using docker file to deploy airflow to RedHat Openshift environment with help of below command RUN pip3 install --trusted-host=files.pythonhosted.org --trusted-host=pypi.org "apache-airflow[crypto,celery,postgres,hive,jdbc,kubernetes,ssh,oracle,aws]==${AIRFLOW_VERSION}" --constraint constraints-3.8.txt --cert root.pem ### Anything else _No response_ ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
