liamkernan opened a new pull request, #61557:
URL: https://github.com/apache/airflow/pull/61557
## Summary
- Fixes `airflow variables import` rejecting JSON/YAML files where a
variable value is a list with multiple items (e.g., `[{"foo": "bar"}, {"bar":
"foo"}]`)
- The duplicate-key check in `load_variables()` was designed for `.env`
files but incorrectly applied to JSON/YAML, where list values are legitimate
variable data
- This caused a round-trip failure: `airflow variables export` could produce
files that `airflow variables import` refused to load
## Details
The `_parse_env_file` parser wraps values in lists when a key appears
multiple times, so `isinstance(values, list) and len(values) != 1` correctly
detects duplicate keys for `.env` files. However, `_parse_json_file` and
`_parse_yaml_file` return raw parsed values, so a list value like `[{"foo":
"bar"}, {"bar": "foo"}]` was incorrectly flagged as "multiple values for a key."
The fix limits the duplicate-key check to `.env` files only and passes
JSON/YAML values through as-is. The `LocalFilesystemBackend._local_variables`
property now serializes non-string values to JSON strings to maintain the `str`
return type contract for the secrets backend interface.
## Test plan
- [x] Added parametrized tests for JSON files with list, int, and bool
variable values
- [x] Added parametrized test for YAML files with list variable values
- [x] All 76 existing + new tests pass
- [x] Pre-commit checks pass (ruff, codespell, bandit, etc.)
Closes #61490
Made with [Cursor](https://cursor.com)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
