WenyXu commented on issue #3725: URL: https://github.com/apache/incubator-opendal/issues/3725#issuecomment-1848330498
> By the way, I also think it's a good idea to implment `parquet::arrow::AsyncFileReader` in a seperate crate. So that opendal users can have best performance. IMO, for database developers, they generally will carefully design the crucial path of fetching/reading. However, there are some other scenarios we also need to consider, such as importing data to the database. These cases typically require supporting multiple file formats, and the size of files are variants(very large or very small). Most readers of these formats may behave similarly to the `ParquetStream`; they read data piece after piece. A general buffered and lazy-preferred-io-fetching reader may be better. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
