etseidl commented on code in PR #10407:
URL: https://github.com/apache/arrow-rs/pull/10407#discussion_r3634841911
##########
arrow-avro/src/reader/vlq.rs:
##########
@@ -29,19 +31,30 @@ pub struct VLQDecoder {
impl VLQDecoder {
/// Decode a signed long from `buf`
- pub fn long(&mut self, buf: &mut &[u8]) -> Option<i64> {
+ ///
+ /// Returns `Err` if `buf` starts with more than 10 continuation bytes
without a
+ /// terminator, which would otherwise overflow the accumulated `u64`
(matching the
+ /// bound already enforced by [`read_varint_array`] for the stateless
decoders below).
+ pub fn long(&mut self, buf: &mut &[u8]) -> Result<Option<i64>, AvroError> {
while let Some(byte) = buf.first().copied() {
+ if self.shift == 63 && byte >= 0x02 {
Review Comment:
Adding more branching to the loop makes me nervous...I'd like to see some
benchmarks for this.
Perhaps something like
```rust
self.in_progress |=
((byte & 0x7F) as u64)
.checked_shl(self.shift)
.ok_or(AvroError::ParseError(
"Malformed Avro varint: too many continuation
bytes".to_string(),
))?;
```
would work?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
