WenyXu commented on code in PR #3734: URL: https://github.com/apache/incubator-opendal/pull/3734#discussion_r1423676321
########## core/src/docs/rfcs/3734_buffered_reader.md: ########## @@ -0,0 +1,113 @@ +- Proposal Name: `buffered_reader` +- Start Date: 2023-12-10 +- RFC PR: [apache/incubator-opendal#3574](https://github.com/apache/incubator-opendal/pull/3734) +- Tracking Issue: [apache/incubator-opendal#3575](https://github.com/apache/incubator-opendal/issues/3735) + +# Summary + +Amortize overhead of IO. + +# Motivation + +The aim is to amortize the overhead of IO. Some scenarios, such as importing data to the database, typically require supporting multiple file formats, and the size of files varies(very large or very small). Most readers of these formats may behave similarly to the `ParquetStream`; they read and decode data piece after piece. + +``` +File +┌───┬───┬───┬──┐ +│ │ │ │ │ +└───┴───┴───┘▲─┘ + │ + │ 1. SeekFromEnd(8) + + 2. ReadToEnd(limit) Review Comment: I removed the redundant concepts and added some figures, which should be easier to understand than before.🥹 ########## core/src/docs/rfcs/3734_buffered_reader.md: ########## @@ -0,0 +1,113 @@ +- Proposal Name: `buffered_reader` +- Start Date: 2023-12-10 +- RFC PR: [apache/incubator-opendal#3574](https://github.com/apache/incubator-opendal/pull/3734) +- Tracking Issue: [apache/incubator-opendal#3575](https://github.com/apache/incubator-opendal/issues/3735) + +# Summary + +Amortize overhead of IO. + +# Motivation + +The aim is to amortize the overhead of IO. Some scenarios, such as importing data to the database, typically require supporting multiple file formats, and the size of files varies(very large or very small). Most readers of these formats may behave similarly to the `ParquetStream`; they read and decode data piece after piece. + +``` +File +┌───┬───┬───┬──┐ +│ │ │ │ │ +└───┴───┴───┘▲─┘ + │ + │ 1. SeekFromEnd(8) + + 2. ReadToEnd(limit) Review Comment: I removed the redundant concepts and added some figures, which should be easier to understand than before.🥹 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
