ValeriiKretinin opened a new issue, #37790:
URL: https://github.com/apache/airflow/issues/37790
### Apache Airflow version
Other Airflow 2 version (please specify below)
### If "Other Airflow 2 version" selected, which one?
2.7.X and 2.8.X
### What happened?
To store secrets and connections we use hashicorp as a secret backend.
After updating airflow from version 2.6.X to 2.7.X and 2.8.X we got the
following problem:
```
[2024-02-29, 09:05:06 UTC] {connection.py:471} ERROR - Unable to retrieve
connection from secrets backend (VaultBackend). Checking subsequent secrets
backend.
Traceback (most recent call last):
File
"/home/airflow/.local/lib/python3.11/site-packages/airflow/models/connection.py",
line 468, in get_connection_from_secrets
SecretCache.save_connection_uri(conn_id, conn.get_uri())
^^^^^^^^^^^^^^
File
"/home/airflow/.local/lib/python3.11/site-packages/airflow/models/connection.py",
line 264, in get_uri
host_block += quote(host, safe="")
^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/urllib/parse.py", line 893, in quote
return quote_from_bytes(string, safe)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/urllib/parse.py", line 923, in
quote_from_bytes
raise TypeError("quote_from_bytes() expected bytes")
TypeError: quote_from_bytes() expected bytes
[2024-02-29, 09:05:06 UTC] {taskinstance.py:1937} ERROR - Task failed with
exception
Traceback (most recent call last):
File
"/home/airflow/.local/lib/python3.11/site-packages/airflow/operators/python.py",
line 263, in execute
condition = super().execute(context)
^^^^^^^^^^^^^^^^^^^^^^^^
File
"/home/airflow/.local/lib/python3.11/site-packages/airflow/operators/python.py",
line 192, in execute
return_value = self.execute_callable()
^^^^^^^^^^^^^^^^^^^^^^^
File
"/home/airflow/.local/lib/python3.11/site-packages/airflow/operators/python.py",
line 209, in execute_callable
return self.python_callable(*self.op_args, **self.op_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/airflow/dags/abnormal_multiples/abnormal_multiples_dag.py",
line 110, in generate_abnormal_multiples_to_post_in_slack
client = init_clickhouse_client(clickhouse_connection_id)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/airflow/dags/common/clickhouse_client.py", line 5, in
init_clickhouse_client
conn = BaseHook.get_connection(clickhouse_connection)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/home/airflow/.local/lib/python3.11/site-packages/airflow/hooks/base.py", line
72, in get_connection
conn = Connection.get_connection_from_secrets(conn_id)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/home/airflow/.local/lib/python3.11/site-packages/airflow/models/connection.py",
line 477, in get_connection_from_secrets
raise AirflowNotFoundException(f"The conn_id `{conn_id}` isn't defined")
airflow.exceptions.AirflowNotFoundException: The conn_id `clickhouse_dwh`
isn't defined
```
Based on the analysis of the latest changes here
https://github.com/apache/airflow/blob/main/airflow/models/connection.py, it
was possible to find out that only the str type was supported, after which list
is no longer supported, accordingly, all connections in vault need to be
converted to str
If change host from list to a string, the error goes away and everything
works correctly again, although this was not required in versions 2.6.X and
older
### What you think should happen instead?
It would be great to get back support for type list, since it is more
convenient to store several hosts in a list rather than in one line or create
different connections to store different hosts
However, if you do not plan to support list or it is technically difficult
to implement, the issue can be closed
### How to reproduce
Create a connection in vault and enter a list of hosts in list
```
host: [
host1,
host2....
]
```
### Operating System
k8s helm chart
### Versions of Apache Airflow Providers
apache-airflow-providers-hashicorp 3.6.3
also tried with 3.5.0
### Deployment
Other 3rd-party Helm chart
### Deployment details
community helm chart
### Anything else?
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
