JavierLopezT edited a comment on pull request #17448: URL: https://github.com/apache/airflow/pull/17448#issuecomment-933282983
Hello, this might sound ridiculous but I am trying to test this MR for https://github.com/apache/airflow/issues/18638#issuecomment-932991258 and I am unable to do it. Could you help me, please? While I was coding the MR and for several months, I have been using the Secrets Backend extending my docker image as follow: ``` COPY plugins/aws_secrets_manager_backend.py /home/airflow/.local/lib/python3.9/site-packages/airflow/providers/amazon/aws/secrets/secrets_manager.py COPY aws_config /home/airflow/.aws/config COPY aws_credentials /home/airflow/.aws/credentials ``` In the code I had the custom defaults I needed, so no need for backend kwargs. Thus, my docker-compose only needed one env variable for having the backend working: ` AIRFLOW__SECRETS__BACKEND: airflow.providers.amazon.aws.secrets.secrets_manager.SecretsManagerBackend` Now, I have pinned in my requirements file the amazon library `apache-airflow-providers-amazon==2.3.0rc1`. I have deleted the COPY command of the secrets file from the Dockerfile and added the kwargs as env variable as follow: ` AIRFLOW__SECRETS__BACKEND_KWARGS: '{"connections_prefix": "data", "full_url_mode": False, "sep": "_"}'` I'm trying to connect to the secret `data_db_airflow_snowflake`. I have tried also with `connections_prefix` as an empty string (and the full name of the connection as an argument). In any case, I am having an error like the following: ``` File "/opt/airflow/plugins/snowflake_hook.py", line 213, in get_pandas_df df = super(SnowflakeHook, self).get_pandas_df(sql, parameters, **kwargs) File "/home/airflow/.local/lib/python3.9/site-packages/airflow/hooks/dbapi.py", line 116, in get_pandas_df with closing(self.get_conn()) as conn: File "/opt/airflow/plugins/snowflake_hook.py", line 124, in get_conn conn_config = self._get_conn_params() File "/opt/airflow/plugins/snowflake_hook.py", line 63, in _get_conn_params conn = self.get_connection(self.snowflake_conn_id) File "/home/airflow/.local/lib/python3.9/site-packages/airflow/hooks/base.py", line 67, in get_connection conn = Connection.get_connection_from_secrets(conn_id) File "/home/airflow/.local/lib/python3.9/site-packages/airflow/models/connection.py", line 379, in get_connection_from_secrets raise AirflowNotFoundException(f"The conn_id `{conn_id}` isn't defined") airflow.exceptions.AirflowNotFoundException: The conn_id `db_snowflake_***` isn't defined ``` Entering in the container, the env variables, as well as the secrets file, seems to be right. Any idea? Thank you very much @mik-laj @potiuk @uranusjr -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
