iemejia commented on PR #3861: URL: https://github.com/apache/avro/pull/3861#issuecomment-5019895781
Thanks for the detailed benchmarking, @arib06. Applied both suggestions: - `_ensure_collection_available` now only calls `bytes_remaining()` when the block count exceeds a `_MAX_UNCHECKED_COLLECTION` threshold (1024), mirroring `_MAX_UNCHECKED_READ` in `read()`; the structural cap stays unconditional with no seek. As you noted this doesn't weaken the bound — with `min_bytes > 0` an under-threshold block still has to be paid for in real wire bytes, and the zero-byte path is untouched and keeps its tighter cumulative cap. Added a regression test for a small truncated collection to lock that in. - Expanded the `_collection_limits()` docstring to call out explicitly that `AVRO_MAX_COLLECTION_ITEMS` pins both limits to the same value, so raising the zero-byte limit also lowers the structural cap (and how to avoid that by setting it above `DEFAULT_MAX_COLLECTION_STRUCTURAL`). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
