zhongzc commented on code in PR #3734: URL: https://github.com/apache/incubator-opendal/pull/3734#discussion_r1422055758
########## core/src/docs/rfcs/3734_buffered_reader.md: ########## @@ -0,0 +1,113 @@ +- Proposal Name: `buffered_reader` +- Start Date: 2023-12-10 +- RFC PR: [apache/incubator-opendal#3574](https://github.com/apache/incubator-opendal/pull/3734) +- Tracking Issue: [apache/incubator-opendal#3575](https://github.com/apache/incubator-opendal/issues/3735) + +# Summary + +Amortize overhead of IO. + +# Motivation + +The aim is to amortize the overhead of IO. Some scenarios, such as importing data to the database, typically require supporting multiple file formats, and the size of files varies(very large or very small). Most readers of these formats may behave similarly to the `ParquetStream`; they read and decode data piece after piece. + +``` +File +┌───┬───┬───┬──┐ +│ │ │ │ │ +└───┴───┴───┘▲─┘ + │ + │ 1. SeekFromEnd(8) + + 2. ReadToEnd(limit) Review Comment: I don't quite understand what this legend is trying to convey 🥲 ########## core/src/docs/rfcs/3734_buffered_reader.md: ########## @@ -0,0 +1,113 @@ +- Proposal Name: `buffered_reader` +- Start Date: 2023-12-10 +- RFC PR: [apache/incubator-opendal#3574](https://github.com/apache/incubator-opendal/pull/3734) +- Tracking Issue: [apache/incubator-opendal#3575](https://github.com/apache/incubator-opendal/issues/3735) + +# Summary + +Amortize overhead of IO. + +# Motivation + +The aim is to amortize the overhead of IO. Some scenarios, such as importing data to the database, typically require supporting multiple file formats, and the size of files varies(very large or very small). Most readers of these formats may behave similarly to the `ParquetStream`; they read and decode data piece after piece. + +``` +File +┌───┬───┬───┬──┐ +│ │ │ │ │ +└───┴───┴───┘▲─┘ + │ + │ 1. SeekFromEnd(8) + + 2. ReadToEnd(limit) Review Comment: I don't quite understand what this legend is trying to convey 🥲 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
