Eason09053360 opened a new pull request, #73083:
URL: https://github.com/apache/airflow/pull/73083
## Why
`airflow connections export --file-format env --serialization-format json`
writes a file that
`airflow connections import` cannot read back. In `_create_connection`, a
string value from a
`.env` file was always treated as a URI, and `urlsplit()` never rejects
anything, so the JSON
payload was parsed as a path: `conn_type`, `host`, `login` and `port` came
out empty and the rest
of the JSON blob landed in the `schema` column. The command printed
`Imported connection <id>` and
exited 0, so the corruption is silent until a task fails to connect.
```console
$ airflow connections export conns.env --file-format env
--serialization-format json
$ airflow connections import conns.env
Imported connection my_conn
```
| | conn_type | host | login | port | schema |
|---|---|---|---|---|---|
| before | *(empty)* | *(empty)* | *(empty)* | *(empty)* | `"conn_type":
"mysql", ... }` |
| after | mysql | myhost | mylogin | 3306 | mysch |
That value is the same string one would put in `AIRFLOW_CONN_*`, and every
other secrets backend
already accepts both forms there:
`BaseSecretsBackend._deserialize_connection_value` treats a
`{`-prefixed value as JSON and anything else as a URI. `local_filesystem`
was the only backend
missing that test.
## What
- `airflow-core/src/airflow/secrets/local_filesystem.py`: a `{`-prefixed
string value is
deserialized with `Connection.from_json()` instead of being parsed as a
URI. A URI can never
start with `{` (RFC 3986 requires the scheme to begin with a letter), so
the two forms are
unambiguous and existing URI files are untouched. Going through
`from_json()` rather than this
module's object branch also keeps `.env` values consistent with
`AIRFLOW_CONN_*` for `conn_type`
normalization and `port` coercion. Malformed JSON, which previously
produced a garbage
connection, now raises a `ValueError` naming the connection.
-
`airflow-core/docs/security/secrets/secrets-backend/local-filesystem-secrets-backend.rst`:
document that a `.env` value may be a URI or a JSON object.
- Tests:
`test_cli_connections_import_should_round_trip_env_file_exported_as_json`
covers the
reported export/import path;
`test_env_file_json_connection_is_normalized_like_an_environment_variable`
pins the
`postgresql` -> `postgres` and `"5432"` -> `5432` parity with the
environment variable backend.
`.json` and `.yaml` files still go through this module's object branch,
which does not normalize
`conn_type` or coerce `port`. That predates this change and is left alone
here — happy to follow
up separately if reviewers want the two paths unified.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
