[
https://issues.apache.org/jira/browse/AVRO-4300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ismaël Mejía updated AVRO-4300:
-------------------------------
Description:
When GenericDatumReader decodes an array it reads the block item count from the
stream and pre-allocates the backing store (newArray -> new Object[count])
before decoding any element. Two independent gaps let a tiny payload drive an
unbounded allocation and exhaust the heap:
1. Zero-byte elements (classic and fast reader).
Elements whose schema encodes to zero bytes (null, a zero-length fixed, or a
record with only zero-byte fields) consume no input, so
ensureAvailableCollectionBytes (AVRO-4241) skips the check for them
(minBytesPerElement == 0). The collection-length cap is Integer.MAX_VALUE-8, a
JVM array-size ceiling rather than a memory budget. So an array such as
{"type":"array","items":"null"} declaring a block count of 200,000,000 is a ~6
byte payload that allocates a 200M-slot array (~1.6 GB).
2. The fast reader never had the AVRO-4241 available-bytes guard at all.
AVRO-4241 only modified the classic GenericDatumReader (and BinaryDecoder /
Decoder / ValidatingDecoder); it never touched FastReaderBuilder, which is the
default decode path (avro.io.fastread defaults to true). As a result, on the
default reader even a non-zero-byte array such as array<long> or array<int>
with a huge block count and no data pre-allocated new GenericData.Array<>((int)
count) and exhausted the heap. Only the classic reader was protected.
Fix (applied identically on both the classic and fast reader paths):
- Add SystemLimitException.checkMaxCollectionAllocation, a heap-aware
cumulative cap for zero-byte elements (default maxMemory()/4/8 elements,
overridable via the org.apache.avro.limits.collectionItems.maxAllocation system
property, mirroring the existing decompression limit).
- Expose GenericDatumReader.ensureAvailableCollectionBytes and apply it,
together with the zero-byte cap, before allocating each array block in
FastReaderBuilder, so the fast and classic readers enforce identical guards.
- Maps were already safe on both paths because each entry carries a string key
of at least one byte (ensureAvailableMapBytes on the classic path; key reads
consume bytes on the fast path).
Verification: a full matrix of array<null|long|int> and map<null|long> at count
200,000,000 under -Xmx256m is rejected on both the fast and classic readers
(SystemLimitException for zero-byte elements, EOFException otherwise) instead
of OutOfMemoryError; legitimate collections within the limit still decode. Unit
tests cover all ten collection/reader combinations plus cumulative multi-block
and positive-decode cases. This completes AVRO-4277 by covering the Java SDK's
residual gaps.
was:
When GenericDatumReader decodes an array or map it reads the block item count
from the stream and pre-allocates the full backing store up front
(newArray/newMap -> new Object[count]) before decoding any element.
The two existing guards do not bound a collection whose elements encode to zero
bytes:
- ensureAvailableCollectionBytes (AVRO-4241) only enforces count *
minBytesPerElement <= remainingBytes when minBytesPerElement > 0.
minBytesPerElement returns 0 for null and for self-referencing records, so the
check is skipped for those element types.
- SystemLimitException.checkMaxCollectionLength caps the count at
Integer.MAX_VALUE - 8, which is a JVM array-size ceiling rather than a memory
budget.
As a result, a small input declaring an array/map of many zero-byte elements
(e.g. {"type":"array","items":"null"} with a block count of 200,000,000) drives
a single large up-front allocation (new Object[200000000], ~1.6 GB) and
exhausts the heap. Confirmed against 1.11.5, 1.12.1 and current main.
Proposed fix: do not pre-size the backing collection to the untrusted block
count; clamp the initial capacity and let the collection grow as elements are
actually decoded, keeping the cumulative checkMaxCollectionLength cap as the
ultimate bound. Add tests and verify the behaviour with a zero-byte-element
reproduction. This completes AVRO-4277 by covering the Java SDK's residual gap.
Summary: [java] Bound array allocation when decoding on both the
classic and fast readers (zero-byte elements + missing fast-reader
available-bytes check) (was: [java] Bound collection allocation for zero-byte
elements when decoding arrays and maps)
> [java] Bound array allocation when decoding on both the classic and fast
> readers (zero-byte elements + missing fast-reader available-bytes check)
> -------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: AVRO-4300
> URL: https://issues.apache.org/jira/browse/AVRO-4300
> Project: Apache Avro
> Issue Type: Sub-task
> Components: java
> Affects Versions: 1.11.5, 1.12.1
> Reporter: Ismaël Mejía
> Assignee: Ismaël Mejía
> Priority: Major
> Fix For: 1.12.2
>
>
> When GenericDatumReader decodes an array it reads the block item count from
> the stream and pre-allocates the backing store (newArray -> new
> Object[count]) before decoding any element. Two independent gaps let a tiny
> payload drive an unbounded allocation and exhaust the heap:
> 1. Zero-byte elements (classic and fast reader).
> Elements whose schema encodes to zero bytes (null, a zero-length fixed, or a
> record with only zero-byte fields) consume no input, so
> ensureAvailableCollectionBytes (AVRO-4241) skips the check for them
> (minBytesPerElement == 0). The collection-length cap is Integer.MAX_VALUE-8,
> a JVM array-size ceiling rather than a memory budget. So an array such as
> {"type":"array","items":"null"} declaring a block count of 200,000,000 is a
> ~6 byte payload that allocates a 200M-slot array (~1.6 GB).
> 2. The fast reader never had the AVRO-4241 available-bytes guard at all.
> AVRO-4241 only modified the classic GenericDatumReader (and BinaryDecoder /
> Decoder / ValidatingDecoder); it never touched FastReaderBuilder, which is
> the default decode path (avro.io.fastread defaults to true). As a result, on
> the default reader even a non-zero-byte array such as array<long> or
> array<int> with a huge block count and no data pre-allocated new
> GenericData.Array<>((int) count) and exhausted the heap. Only the classic
> reader was protected.
> Fix (applied identically on both the classic and fast reader paths):
> - Add SystemLimitException.checkMaxCollectionAllocation, a heap-aware
> cumulative cap for zero-byte elements (default maxMemory()/4/8 elements,
> overridable via the org.apache.avro.limits.collectionItems.maxAllocation
> system property, mirroring the existing decompression limit).
> - Expose GenericDatumReader.ensureAvailableCollectionBytes and apply it,
> together with the zero-byte cap, before allocating each array block in
> FastReaderBuilder, so the fast and classic readers enforce identical guards.
> - Maps were already safe on both paths because each entry carries a string
> key of at least one byte (ensureAvailableMapBytes on the classic path; key
> reads consume bytes on the fast path).
> Verification: a full matrix of array<null|long|int> and map<null|long> at
> count 200,000,000 under -Xmx256m is rejected on both the fast and classic
> readers (SystemLimitException for zero-byte elements, EOFException otherwise)
> instead of OutOfMemoryError; legitimate collections within the limit still
> decode. Unit tests cover all ten collection/reader combinations plus
> cumulative multi-block and positive-decode cases. This completes AVRO-4277 by
> covering the Java SDK's residual gaps.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
