iemejia commented on code in PR #3865:
URL: https://github.com/apache/avro/pull/3865#discussion_r3567515815
##########
lang/java/avro/src/main/java/org/apache/avro/io/BinaryDecoder.java:
##########
@@ -458,7 +466,7 @@ public long arrayNext() throws IOException {
@Override
public long skipArray() throws IOException {
- return doSkipItems();
+ return SystemLimitException.checkMaxCollectionLength(doSkipItems());
}
Review Comment:
Fixed — doSkipItems now rejects a Long.MIN_VALUE block count (matching
doReadItemCount) and a negative block byte-size, instead of silently continuing
to skip.
##########
lang/java/avro/src/main/java/org/apache/avro/generic/GenericDatumReader.java:
##########
@@ -114,6 +115,26 @@ public void setExpected(Schema reader) {
private static final ThreadLocal<Map<Schema, Map<Schema, ResolvingDecoder>>>
RESOLVER_CACHE = ThreadLocalWithInitial
.of(WeakIdentityHashMap::new);
+ /**
+ * Upper bound on the initial capacity eagerly allocated for a collection
from
+ * its declared block count. The backing array/map grows on demand as
elements
+ * are read, so this is only a starting hint: it prevents a large declared
count
+ * from driving a huge up-front allocation before any element is decoded.
This
Review Comment:
This is intentional. The clamp is only the *initial* capacity; the array/map
still grows to hold every element actually read, and the growth is geometric
(amortized O(n)), so for a legitimate large in-memory array it costs a handful
of extra resizes rather than a single up-front allocation. Keeping it
unconditional keeps the hot path simple and also covers the cases where it's
essential — zero-byte elements (bounded by the 10M item cap, not by bytes, so
remainingBytes doesn't constrain them) and stream sources (remainingBytes() ==
-1). A remaining/zero-byte-aware refinement could be added later if profiling
shows the extra resizes matter, but functionally this is safe and the perf
impact is negligible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
