zeroshade commented on issue #37: URL: https://github.com/apache/arrow-go/issues/37#issuecomment-5501445110
Closing this as stale — I asked on 2026-07-11 whether the slowness still reproduced after the many reader performance improvements since 2024, and we didn't hear back. For anyone landing here with the same question: pass an `io.ReaderAt` that does S3 ranged GETs (for example https://github.com/wolfeidau/s3iofs) to `file.NewParquetReader`. Dropping to `file.NewPageReader` yourself is not worth it. If throughput is poor, the things worth checking are `ReaderProperties.BufferedStreamEnabled`, reading only the row groups and columns you need, and whether the client is doing many small ranged reads rather than a few large ones. @Zeeyi13 if 12 minutes for a 140MB file is still what you see on a current release, please open a new issue with the reader setup and I'll dig into where the time is going — that number is far worse than expected and I'd like to understand it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
