iemejia opened a new pull request, #3929:
URL: https://github.com/apache/avro/pull/3929
## What is the purpose of the change
Python SDK implementation of AVRO-4302 (parent). Recursive schemas (e.g. a
linked list or tree) let a small, hostile payload drive arbitrarily deep
nesting during binary decoding, exhausting the Python call stack
(`RecursionError`, or a fatal interpreter crash once the C stack is
exhausted)
before any allocation limit is reached.
This bounds the decode nesting depth by counting structural descents into
records, arrays, maps and unions in `DatumReader.read_data`, rejecting input
that nests deeper than the limit with a bounded `AvroException` instead of a
`RecursionError` / crash. The default is 100 (matching Protocol Buffers and
the
Java SDK) and is configurable via the `AVRO_MAX_DECODE_DEPTH` environment
variable. The depth is tracked per `DatumReader` for the current datum,
reset at
the start of each top-level `read()`, and restored on exit so a reader can be
reused.
## Verifying this change
This change added tests and can be verified as follows:
- Added `test_decode_recursion_depth.py`: a ~100k-deep recursive linked-list
payload is rejected with a bounded `AvroException` (not a
`RecursionError`),
a moderately nested value within the limit still decodes, and the
`AVRO_MAX_DECODE_DEPTH` override is honored.
- `ruff check`, `ruff format` and `mypy --strict` pass.
## Documentation
- Does this pull request introduce a new feature? (no — DoS hardening)
- If yes, how is the feature documented? (docstrings — the new
`AVRO_MAX_DECODE_DEPTH` limit is documented alongside the existing
collection
limits in `avro/io.py`)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
