kacpermuda opened a new issue, #58386: URL: https://github.com/apache/airflow/issues/58386
### Apache Airflow Provider(s) databricks ### Versions of Apache Airflow Providers apache-airflow-providers-databricks==7.7.4 ### Apache Airflow version Tested on both latest AF2 and 3 ### Operating System ubuntu ### Deployment Astronomer ### Deployment details _No response_ ### What happened `databricks-sql-connector==4.2.0` released Friday November 14th introduced [changes](https://github.com/databricks/databricks-sql-python/pull/704/files#diff-9a357d81b1ee1590c59205332b013554cc994fce904f272e6a9b88ed58ffe707R517) to commit/autocommit properties on Databricks Connection object. Because of that changes, when using Databricks operator like `DatabricksSqlOperator` I'm receiving an error: `databricks.sql.exc.TransactionError: Failed to set autocommit to False: [CONFIG_NOT_AVAILABLE] Configuration AUTOCOMMIT is not available. SQLSTATE: 42K0I`. Regardless of whether I try to use autocommit=False or True, the problem is that the Databricks is not accepting setting this config with SQL query. The Base DbApiHook [set_autocomit](https://github.com/apache/airflow/blob/main/providers/common/sql/src/airflow/providers/common/sql/hooks/sql.py#L827) method is setting the autocommit on dbx conn object, before it did not have this property so it passed and was ignored, but now that's i mplemented, it started to fail. ### What you think should happen instead I should be able to execute query on DBX without errors. ### How to reproduce Run any query with `DatabricksSqlOperator` and `databricks-sql-connector==4.2.0`. ### Anything else Not sure if the problem is within my DBX instance, if we should somehow work around this in Airflow or maybe the changes to databricks connection are to be adjusted. ### Are you willing to submit PR? - [x] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
