Copilot commented on code in PR #3861:
URL: https://github.com/apache/avro/pull/3861#discussion_r3567610879
##########
lang/py/avro/io.py:
##########
@@ -795,23 +981,59 @@ def read_array(self, writers_schema:
avro.schema.ArraySchema, readers_schema: av
The actual count in this case
is the absolute value of the count written.
"""
- read_items = []
+ read_items: List[object] = []
+ min_bytes = _min_bytes_per_element(writers_schema.items)
+ zero_byte_limit, structural_limit = _collection_limits()
block_count = decoder.read_long()
while block_count != 0:
if block_count < 0:
block_count = -block_count
decoder.skip_long()
Review Comment:
For negative block counts, the following block-size long is currently
skipped via `decoder.skip_long()`. That bypasses the new `read_long()`
varint-length validation (and will scan arbitrarily long malformed varints
byte-by-byte), so a crafted stream can still cause excessive work here. Read
and validate the block-size with `read_long()` (even if you ignore the value)
so malformed/overlong varints are rejected consistently.
##########
lang/py/avro/io.py:
##########
@@ -832,24 +1054,43 @@ def read_map(self, writers_schema:
avro.schema.MapSchema, readers_schema: avro.s
The actual count in this case
is the absolute value of the count written.
"""
- read_items = {}
+ read_items: Dict[str, object] = {}
+ # Map keys are strings (>= 1 byte length prefix) plus the value.
+ min_bytes = 1 + _min_bytes_per_element(writers_schema.values)
+ zero_byte_limit, structural_limit = _collection_limits()
+ # Track decoded pairs separately: len(read_items) counts unique keys,
so
+ # duplicate keys (later entries overwrite earlier ones) would
undercount
+ # and let a multi-block map exceed the cumulative caps.
+ items_read = 0
block_count = decoder.read_long()
while block_count != 0:
if block_count < 0:
block_count = -block_count
decoder.skip_long()
Review Comment:
For negative map block counts, the block-size long is skipped via
`decoder.skip_long()`, which bypasses `read_long()`’s overlong-varint checks
and can scan arbitrarily long malformed varints. Read (and minimally validate)
the block size with `read_long()` so malformed encodings are rejected
consistently and can’t drive excessive work.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
