WenyXu commented on code in PR #3734: URL: https://github.com/apache/incubator-opendal/pull/3734#discussion_r1424024163
########## core/src/docs/rfcs/3734_buffered_reader.md: ########## @@ -0,0 +1,150 @@ +- Proposal Name: `buffered_reader` +- Start Date: 2023-12-10 +- RFC PR: [apache/incubator-opendal#3574](https://github.com/apache/incubator-opendal/pull/3734) +- Tracking Issue: [apache/incubator-opendal#3575](https://github.com/apache/incubator-opendal/issues/3735) + +# Summary + +Allowing the underlying reader to fetch data at the buffer's size to amortize the IO's overhead. + +# Motivation + +The objective is to mitigate the IO overhead. In certain scenarios, the reader processes the data incrementally, meaning that it utilizes the `seek()` function to navigate to a specific position within the file. Subsequently, it invokes the `read()` to reads `limit` bytes into memory and performs the decoding process. + +``` +File Review Comment: > Could you elaborate on how a buffer will be beneficial in this scenario? > > ```rust > let r = op.reader(path).await; > r.seek(SeekFrom::End(-8)).await; > r.read_to_end(&Vec::new()).await; > ``` > > Seems buffer is not helpful. Oh, I misunderstood before; in this scenario, invoking `r.read_to_end(& Vec: :new ()).await`; will execute a read of the file within the range (-8, end). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
