Christian created ARROW-15729:
---------------------------------
Summary: Reading large files randomly freezes
Key: ARROW-15729
URL: https://issues.apache.org/jira/browse/ARROW-15729
Project: Apache Arrow
Issue Type: Bug
Components: R
Reporter: Christian
Fix For: 6.0.1
Hi -
I recently upgraded to Arrow 6.0.1 and am using it in R.
Whenever reading a large file (~10gb) in Windows it randomly freezes sometimes.
I can see the memory being allocated in the first 10-20 seconds, but then
nothing happens and R just doesn't respond.
I'm using the option options(arrow.use_threads=FALSE).
I didn't have this issue with the previous version I was using. And the file
reads fine under Linux.
I would post a reproducible example but it happens randomly. I even thought I
would just read large files in pieces by first getting all the distinct
sections of a specific column (with compute>collect) but that hangs too.
Any ideas would be appreciated.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
