xgbtxy opened a new pull request, #4086:
URL: https://github.com/apache/avro/pull/4086
## What is the purpose of the change
Fix two coupled specification-compliance bugs in
`DatumReader._read_default_value` (`lang/py/avro/io.py`) during schema
evolution:
1. **Nested `RecordSchema` explicit `null` defaults**: `_read_default_value`
previously evaluated `json_val = default_value.get(field.name); if json_val is
None: json_val = field.default`. When an outer record's `default` JSON object
explicitly set an inner nullable field to `null` (e.g., `"default": {"x":
null}` where inner field `x` has `"type": ["int", "null"], "default": 99`),
`json_val is None` conflated the explicit `null` with an omitted key and
silently overwrote `None` with `99`. Conversely, omitting a required inner
field (`has_default == False`) whose type started with `"null"` silently
fabricated `None` instead of raising `InvalidDefaultException`.
2. **`UnionSchema` first-matching-branch resolution**: Per the [Apache Avro
Specification (Schema Declaration —
Records)](https://avro.apache.org/docs/current/specification/#schema-record),
*"Default values for union fields correspond to the first schema that matches
in the union."* `_read_default_value` previously hardcoded
`field_schema.schemas[0]`, rejecting valid defaults matching subsequent union
branches (including `null` in `["int", "null"]`). Resolving against the first
matching branch also requires strict JSON type guards on `boolean`,
`int`/`long` (`not isinstance(..., bool)` and 32/64-bit bounds), `EnumSchema`
symbol membership, and `ArraySchema` (`not isinstance(..., (str, bytes,
Mapping))`) so earlier branches do not falsely coerce incompatible JSON
defaults.
No JIRA issue: the ASF Jira tracker requires an account.
## Verifying this change
This change added tests and can be verified as follows:
- Added `TestDefaultValueResolution` in `lang/py/avro/test/test_io.py`
covering explicit `null` in nested record defaults (`{"x": None}` vs `{}`),
omitted inner fields with `default: null`, rejection of omitted required inner
fields, union defaults matching non-first primitive/enum/array/record branches,
and rejection of unmatched union defaults.
- Verified locally with `uv run --frozen ruff format --diff .`, `uv run
--frozen ruff check .`, `uv run --frozen mypy avro/`, and `uv run --frozen
python -m unittest avro.test.test_io` (`196` tests passing).
## Documentation
- Does this pull request introduce a new feature? no
- If yes, how is the feature documented? not applicable
Assisted by Google Antigravity.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]