tustvold commented on code in PR #3677:
URL: https://github.com/apache/arrow-rs/pull/3677#discussion_r1102730781
##########
arrow-csv/src/reader/mod.rs:
##########
@@ -438,10 +438,10 @@ impl<R: BufRead> BufReader<R> {
loop {
let buf = self.reader.fill_buf()?;
let decoded = self.decoder.decode(buf)?;
- if decoded == 0 {
+ self.reader.consume(decoded);
+ if decoded == 0 || self.decoder.capacity() == 0 {
Review Comment:
Nope, it will yield only if it has fully read the batch size number of rows.
I.e it will yield if it has read enough data, instead of looping around and
calling `fill_buf` again to potentially read more data that it is just going to
ignore (as it has already read batch_size rows)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
