westonpace commented on issue #34460: URL: https://github.com/apache/arrow/issues/34460#issuecomment-1457015836
There is a way to sort of do this today with `FileReader::GetRecordBatchGenerator`. However, it was one of our first attempts at async support and I think it duplicates a lot of code. What you are describing here (`WillNeed` and `Decode`) sounds more similar to what we ended up adopting in the IPC reader (e.g. `PreBufferMetadata`). Note: what have you have right now is not tested and so it's a bit tricky to understand how exactly you plan to use it. It's not clear to me how you would use this without blocking a CPU thread or doing CPU on an I/O thread. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
