BCantos17 commented on issue #43517: URL: https://github.com/apache/airflow/issues/43517#issuecomment-2450521655
Hi @gopidesupavan, I appreciate the response but I do not think updating to the latest release is a proper fix. I would need to get approval from the rest of the staff unless it's a patch version which I am already on the latest for 2.8.X I did however find more details during my troubleshooting. It has more to with jinja templating than the CNCF provider. We had to use the parameter `template_space` on the SparkKubernetesOperator to pass along the SparkApplication manifest file which required some custom logic involving calling jinja functions. Something here were awry and I switched to using `application_file` to pass long the manifest file. I have no clue why adding in this one spark configuration caused the template rendering logic to fail but my assumption is there are some hardcoded keyword lookup in the jinja library (i hope this isnt the case) Closing the issue as I have found a work around -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
