fhoda edited a comment on issue #14592: URL: https://github.com/apache/airflow/issues/14592#issuecomment-856125981
As of now it seems the expected behavior is not what is happening and is inconsistent across different secret backends. I have tried to reproduce this issue with Airflow 2.0 (main branch) and am not able to do so for any AWS secrets backends. I was only able to reproduce a crashing webserver for GCP Secret Manager and not any other secrets backend. The GCP Secret Manager error seems more to do with the function to get the credentials and not the actual connection. ``` google.auth.exceptions.DefaultCredentialsError: Could not automatically determine credentials. Please set GOOGLE_APPLICATION_CREDENTIALS or explicitly create credentials and re-run the application. For more information, please see https://cloud.google.com/docs/authenticati│google.auth.exceptions.DefaultCredentialsError: Could not automatically determine credentials. Please set GOOGLE_APPLICATION_CREDENTIALS or explicitly create credentials and re-run the application. For more information, please see https://cloud.google.com/docs/authentica on/getting-started ``` I used the `airflow.providers.*` secrets packages for each. I noticed that the original post on the issue uses the `contrib` package and Airflow `1.10.12`. ``` #export AIRFLOW__SECRETS__BACKEND=airflow.providers.hashicorp.secrets.vault.VaultBackend #export AIRFLOW__SECRETS__BACKEND_KWARGS='{"connections_path": "airflow/connections", "variables_path": "airflow/variables", "config_path": "airflow/config", "url": "http://127.0.0.1:8200";, "token": "$VAULT_TOKEN"}' export AIRFLOW__SECRETS__BACKEND=airflow.providers.google.cloud.secrets.secret_manager.CloudSecretManagerBackend export AIRFLOW__SECRETS__BACKEND_KWARGS='{"connections_prefix": "airflow-connections", "variables_prefix": "airflow-variables", "gcp_keyfile_dict": $GCP_SECRET_MANAGER_SA_KEY}' #export AIRFLOW__SECRETS__BACKEND=airflow.providers.microsoft.azure.secrets.azure_key_vault.AzureKeyVaultBackend #export AIRFLOW__SECRETS__BACKEND_KWARGS='{"connections_prefix": "airflow-connections", "variables_prefix": null, "vault_url": "https://example-akv-resource-name.vault.azure.net/"}' #export AIRFLOW__SECRETS__BACKEND=airflow.providers.amazon.aws.secrets.secrets_manager.SecretsManagerBackend #export AIRFLOW__SECRETS__BACKEND_KWARGS='{"connections_prefix": "airflow/connections", "variables_prefix": "airflow/variables", "profile_name": "default"}' #export AIRFLOW__SECRETS__BACKEND=airflow.providers.amazon.aws.secrets.systems_manager.SystemsManagerParameterStoreBackend #export AIRFLOW__SECRETS__BACKEND_KWARGS='{"connections_prefix": "/airflow/connections", "variables_prefix": "/airflow/variables", "profile_name": "default"}' ``` Here are my findings: - AWS Secret Manager - No Crash - AWS SSM - No Crash - Vault - No Crash - Azure Key Vault - No Crash - **GCP Secret Manager - Crash** I believe we should evaluate what the expected behavior should be as compared to what is actually happening. Also after discussing with @kaxil there may be a middle ground for fail over implementation that could make sense here. 1. If configs are being retrieved through the secrets backend then a failure makes sense. 2. If connections and/or variables are not able to be retrieved, then fail over could be a strategy used by users to ensure DAG/task success and predictable execution. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
