dv-gorasiya commented on PR #61654: URL: https://github.com/apache/airflow/pull/61654#issuecomment-3977592355
Hey @shahar1, no worries about the delay! I went ahead and set up a real GCP environment to test this properly. Here's what I did: **Setup:** - Created a fresh GCP project (`airflow-pr-test-61654`) with Secret Manager, BigQuery, and GCS APIs enabled - Configured ADC **without** a quota project so `google.auth.default()` returns `project_id=None` — this is the exact condition that triggers the regression - Unset `GOOGLE_CLOUD_PROJECT` env var and `gcloud config core/project` to make sure nothing leaks a project ID - Created a test secret in Secret Manager - Installed Airflow 3.0.6 + providers-google, patched `credentials_provider.py` to match current `main` branch behavior I wrote 3 test DAGs that each pass an explicit `project_id` and hit real GCP APIs: 1. **`test_secret_manager_backend`** — `CloudSecretManagerBackend(project_id=...)` + reads a secret 2. **`test_bigquery_hook`** — `get_credentials_and_project_id()` + runs a BigQuery query 3. **`test_gcs_hook`** — `get_credentials_and_project_id()` + lists GCS buckets --- ### Before fix (current `main`) — all 3 DAGs fail `_get_credentials_using_adc()` raises before the explicit `project_id` ever gets a chance to be applied. **Secret Manager:**  **BigQuery:**  **GCS:**  --- ### After fix (this PR) — all 3 DAGs pass The explicit `project_id` is now honored. Real secrets are read, real BQ queries run, real GCS calls go through. **Secret Manager:**  **BigQuery:**  **GCS:**  --- Everything ran against the live GCP project. Let me know if you'd like me to test any other operators or scenarios! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
