iemejia commented on code in PR #3859:
URL: https://github.com/apache/avro/pull/3859#discussion_r3567467430
##########
lang/c++/impl/BinaryDecoder.cc:
##########
@@ -177,9 +242,27 @@ size_t BinaryDecoder::skipArray() {
for (;;) {
auto r = doDecodeLong();
if (r < 0) {
- auto n = static_cast<size_t>(doDecodeLong());
- in_.skipBytes(n);
+ auto byteSize = doDecodeLong();
+ if (byteSize < 0) {
+ // A negative block byte-size would convert to a huge size_t
and
+ // drive an unbounded skip; reject it.
+ throw Exception("Invalid negative block size: {}", byteSize);
+ }
+ in_.skipBytes(static_cast<size_t>(byteSize));
} else {
+ // Bound the block count: skipping a huge block of zero-byte
elements
+ // would otherwise loop unboundedly (a CPU exhaustion) even though
it
+ // reads/allocates nothing. The decoder has no element schema
here, so
+ // apply the structural cap (AVRO_MAX_COLLECTION_ITEMS, default
+ // Integer.MAX_VALUE - 8). Read the limit each call so a runtime
+ // change to the environment variable is honoured, matching
Generic.cc.
+ const int64_t structural = maxCollectionStructural();
+ if (r > structural) {
+ throw Exception(
+ "Cannot skip a collection of more than {} elements; "
+ "set AVRO_MAX_COLLECTION_ITEMS if this is legitimate",
+ structural);
+ }
return static_cast<size_t>(r);
Review Comment:
Fixed — skipArray now applies the same `if constexpr (sizeof(size_t) <
sizeof(int64_t))` size_t-fit guard as doDecodeItemCount before casting the
block count, so a count above SIZE_MAX (e.g. from a large env override on
32-bit) is rejected rather than truncated.
##########
lang/c++/include/avro/Stream.hh:
##########
@@ -345,6 +356,28 @@ struct StreamReader {
return next_ != end_ || fill();
}
+ /**
+ * Returns the number of bytes still available to be read: those already
+ * buffered in this reader plus whatever the underlying stream reports as
+ * remaining. Returns a negative value when the underlying stream cannot
+ * report its remaining size.
+ */
+ int64_t remainingBytes() const {
+ if (in_ == nullptr) {
+ return -1;
+ }
+ int64_t streamRemaining = in_->remainingBytes();
+ if (streamRemaining < 0) {
+ return -1;
+ }
+ // Bytes already buffered in this reader, added to what the underlying
+ // stream still has. When next_ and end_ are both null (right after
+ // init()/reset(), before any data is buffered), the subtraction is
+ // well-defined and yields 0.
+ int64_t buffered = end_ - next_;
Review Comment:
This subtraction was previously guarded, but a maintainer review
(standardizing with the rest of Stream.hh) asked for the guard to be removed;
next_ and end_ are always assigned together (both null right after
init()/reset(), both valid otherwise), so in the null case the result is 0 in
practice. Leaving as-is per that earlier review to avoid churn/conflicting
guidance; the comment documents the null case.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
