westonpace commented on issue #13403: URL: https://github.com/apache/arrow/issues/13403#issuecomment-1206622326
> because the dataset always core dump when the program exits Hmm, that's too bad. Any chance you can share a reproducible example of this? > but now I read S3 parquet in a different way If you're only reading a single file and the file fits comfortably in memory then the code you posted is probably ok. It won't overlap compute with I/O (e.g. it will fully finish the I/O before it does any compute) but if your file has only a single row group then that is inevitable at the moment. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
