EB80 commented on issue #11665:
URL: https://github.com/apache/arrow/issues/11665#issuecomment-969113711
Thanks for the reply,
Additional information:
-The feather was written with feather::write_feather and not
arrow::write_feather
-I receive the same error with older versions of arrow (currently running
6.0.0.2)
-I actually encounter a separate issue attempting to write with
arrow::write_feather. 78MB of the 32GB file will write, and then the writing
hangs indefinitely without continuing or crashing. (I used
feather::write_feather for this reason.)
-I am writing to a network drive and not a local disk
-The dataframe is 26M rows and 150 columns
Traceback:
6: stop(e)
5: value[[3L]](cond)
4: tryCatchOne(expr, names, parentenv, handlers[[1L]])
3: tryCatchList(expr, classes, parentenv, handlers)
2: tryCatch(reader$Read(columns), error = read_compressed_error)
1: arrow::read_feather("Archive/2021-06-23/2021-06-23 - Pre-Processed
DECKPLATE - June 2021.feather")
Thank you,
Edward
Edward Beall, PhD
Institute for Defense Analyses
Operational Evaluation Division
Office: (703)845-2463
Cell: (443)277-2648
***@***.******@***.***>
***@***.******@***.***>
From: Joris Van den Bossche ***@***.***>
Sent: Monday, November 15, 2021 8:28 AM
To: apache/arrow ***@***.***>
Cc: Beall, Edward J ***@***.***>; Mention ***@***.***>
Subject: [EXT] Re: [apache/arrow] read_feather error with 32 GB file (Issue
#11665)
*** This email originated outside of IDA. Please verify that you recognize
the sender and know the content is safe before proceeding. ***
@EB80<https://github.com/EB80> Thanks for the report!
Would you be able to share some more info? (for example the full traceback,
how you created the file, whether you can read it with an older version, and if
so what's the schema is, ..?) Or if possible provide a reproducible example?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on
GitHub<https://github.com/apache/arrow/issues/11665#issuecomment-968909862>, or
unsubscribe<https://github.com/notifications/unsubscribe-auth/AWNWW7752TVCITQOBYR3XNDUMEDEJANCNFSM5HYQCQ7Q>.
Triage notifications on the go with GitHub Mobile for
iOS<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
or
Android<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
