WenyXu commented on code in PR #3734: URL: https://github.com/apache/incubator-opendal/pull/3734#discussion_r1423954377
########## core/src/docs/rfcs/3734_buffered_reader.md: ########## @@ -0,0 +1,150 @@ +- Proposal Name: `buffered_reader` +- Start Date: 2023-12-10 +- RFC PR: [apache/incubator-opendal#3574](https://github.com/apache/incubator-opendal/pull/3734) +- Tracking Issue: [apache/incubator-opendal#3575](https://github.com/apache/incubator-opendal/issues/3735) + +# Summary + +Allowing the underlying reader to fetch data at the buffer's size to amortize the IO's overhead. + +# Motivation + +The objective is to mitigate the IO overhead. In certain scenarios, the reader processes the data incrementally, meaning that it utilizes the `seek()` function to navigate to a specific position within the file. Subsequently, it invokes the `read()` to reads `limit` bytes into memory and performs the decoding process. + +``` +File Review Comment: > scenario Yes, In this scenario, the buffer isn’t helpful. I think we might need something called tailing buffer. ########## core/src/docs/rfcs/3734_buffered_reader.md: ########## @@ -0,0 +1,150 @@ +- Proposal Name: `buffered_reader` +- Start Date: 2023-12-10 +- RFC PR: [apache/incubator-opendal#3574](https://github.com/apache/incubator-opendal/pull/3734) +- Tracking Issue: [apache/incubator-opendal#3575](https://github.com/apache/incubator-opendal/issues/3735) + +# Summary + +Allowing the underlying reader to fetch data at the buffer's size to amortize the IO's overhead. + +# Motivation + +The objective is to mitigate the IO overhead. In certain scenarios, the reader processes the data incrementally, meaning that it utilizes the `seek()` function to navigate to a specific position within the file. Subsequently, it invokes the `read()` to reads `limit` bytes into memory and performs the decoding process. + +``` +File Review Comment: > scenario Yes, In this scenario, the buffer isn’t helpful. I think we might need something called tailing buffer. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
