code1704 commented on issue #13827: URL: https://github.com/apache/arrow/issues/13827#issuecomment-1209713919
> I think we did some investigation of partial reads in #11616 but I can't remember if we enabled it for the synchronous path or just the asynchronous path (`GetRecordBatchGenerator`). > > Can you try calling `PreBufferMetadata` before you start reading the record batches? If that doesn't work can you try `GetRecordBatchGenerator`? Thanks for your suggestions. We do not know much about arrow and did not use those APIs you mentioned. Most of the time we use pyarrow to read/write data and do AI model training. So if there's python way, that would be great. We are trying to read the arrow code and try.. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
