yehoshuadimarsky opened a new issue #20660: URL: https://github.com/apache/airflow/issues/20660
### Apache Airflow Provider(s) google ### Versions of Apache Airflow Providers ``` apache-airflow-providers-amazon==2.3.0 apache-airflow-providers-celery==2.1.0 apache-airflow-providers-cncf-kubernetes==2.0.0 apache-airflow-providers-docker==2.2.0 apache-airflow-providers-elasticsearch==2.0.3 apache-airflow-providers-ftp==2.0.1 apache-airflow-providers-google==6.0.0 apache-airflow-providers-grpc==2.0.1 apache-airflow-providers-hashicorp==2.1.1 apache-airflow-providers-http==2.0.1 apache-airflow-providers-imap==2.0.1 apache-airflow-providers-microsoft-azure==3.2.0 apache-airflow-providers-mysql==2.1.1 apache-airflow-providers-odbc==2.0.1 apache-airflow-providers-postgres==2.3.0 apache-airflow-providers-redis==2.0.1 apache-airflow-providers-sendgrid==2.0.1 apache-airflow-providers-sftp==2.1.1 apache-airflow-providers-slack==4.1.0 apache-airflow-providers-sqlite==2.0.1 apache-airflow-providers-ssh==2.2.0 ``` ### Apache Airflow version 2.2.1 ### Operating System Debian Buster ### Deployment Official Apache Airflow Helm Chart ### Deployment details Running the official Airflow Helm chart on GKE. Have Workload Identity set up and working, linked the Google and Kubernetes service accounts. Created the an Airflow connection for GCP ADC, per the [instructions](https://airflow.apache.org/docs/apache-airflow-providers-google/6.2.0/connections/gcp.html#note-on-application-default-credentials) ```bash airflow connections add \ --conn-type google_cloud_platform \ 'gcp-airflow-svc-acct-dev' ``` The SQL connection - note, no password! ```bash airflow connections add \ --conn-type gcpcloudsql \ --conn-host $SQL_INSTANCE_PUBLIC_IP \ --conn-login {GSA_NAME}@{PROJECT_ID}.iam \ # linked to KSA via WI --conn-extra '{"instance": "INSTANCE_NAME", "location": "us-east1", "database_type": "postgres", "project_id": "PROJECT_ID", "use_proxy": true, "sql_proxy_use_tcp": true}' \ 'gcp-sql-ods-dev' ``` ### What happened Getting this error when trying to run a [CloudSQLExecuteQueryOperator](https://airflow.apache.org/docs/apache-airflow-providers-google/6.0.0/_api/airflow/providers/google/cloud/operators/cloud_sql/index.html#airflow.providers.google.cloud.operators.cloud_sql.CloudSQLExecuteQueryOperator) ``` [2022-01-04, 18:03:49 EST] {taskinstance.py:1703} ERROR - Task failed with exception Traceback (most recent call last): File "/home/airflow/.local/lib/python3.9/site-packages/airflow/models/taskinstance.py", line 1332, in _run_raw_task self._execute_task_with_callbacks(context) File "/home/airflow/.local/lib/python3.9/site-packages/airflow/models/taskinstance.py", line 1458, in _execute_task_with_callbacks result = self._execute_task(context, self.task) File "/home/airflow/.local/lib/python3.9/site-packages/airflow/models/taskinstance.py", line 1514, in _execute_task result = execute_callable(context=context) File "/home/airflow/.local/lib/python3.9/site-packages/airflow/providers/google/cloud/operators/cloud_sql.py", line 1076, in execute connection = hook.create_connection() File "/home/airflow/.local/lib/python3.9/site-packages/airflow/providers/google/cloud/hooks/cloud_sql.py", line 932, in create_connection uri = self._generate_connection_uri() File "/home/airflow/.local/lib/python3.9/site-packages/airflow/providers/google/cloud/hooks/cloud_sql.py", line 893, in _generate_connection_uri raise AirflowException("The password parameter needs to be set in connection") airflow.exceptions.AirflowException: The password parameter needs to be set in connection ``` ### What you expected to happen I expect to be able to use the Cloud SQL Auth proxy from GKE using the IAM database authentication of the Workload Identity-provided GSA and not needing any passwords at all. However, the Airflow Hook is hardcoded to raise an error if there is no password in the SQL connection. Why? Outside of Airflow it can work without a password via Cloud SQL Auth proxy and WI etc, can we allow that in Airflow too? ### How to reproduce _No response_ ### Anything else _No response_ ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
