alamb commented on PR #7613: URL: https://github.com/apache/arrow-rs/pull/7613#issuecomment-3197912003
I believe we will support a similar usecase for the reader here: - https://github.com/apache/arrow-rs/issues/7983 The PR is here should you want to follow along - https://github.com/apache/arrow-rs/pull/7997 Basically we will provide a parquet decoder that is not tied to the particular IO I think you can do something similar already for writing using https://docs.rs/parquet/latest/parquet/arrow/arrow_writer/struct.ArrowColumnWriter.html thought that may require buffering in a writer / using channels Maybe we should look into making the split between IO and CPU clearer in the writer too -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
