[
https://issues.apache.org/jira/browse/ARROW-6582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16933622#comment-16933622
]
John Cassil commented on ARROW-6582:
------------------------------------
Thank you very much for your thoughtful responses and the solution to my
original goal.
That might be an interesting approach. If there were a function to export it
from Arrow into text on disk, then a user could essentially use Arrow as a
method to unpack/uncompress a parquet file into raw text. I could see that
being helpful in other situations beyond this. I'm not sure. Your call!
> R's read_parquet() fails with embedded nuls in strings
> ------------------------------------------------------
>
> Key: ARROW-6582
> URL: https://issues.apache.org/jira/browse/ARROW-6582
> Project: Apache Arrow
> Issue Type: Bug
> Components: R
> Affects Versions: 0.14.1
> Environment: Windows 10
> R 3.4.4
> Reporter: John Cassil
> Priority: Major
>
> Apologies if this issue isn't categorized or documented appropriately.
> Please be gentle! :)
> As a heavy R user that normally interacts with parquet files using SparklyR,
> I have recently decided to try to use arrow::read_parquet() on a few parquet
> files that were on my local machine rather than in hadoop. I was not able to
> proceed after several various attempts due to embedded nuls. For example:
> try({df <- read_parquet('out_2019-09_data_1.snappy.parquet') })
> Error in Table__to_dataframe(x, use_threads = option_use_threads()) :
> embedded nul in string: 'INSTALL BOTH LEFT FRONT AND RIGHT FRONT TORQUE
> ARMS\0 ARMS'
> Is there a solution to this?
> I have also hit roadblocks with embedded nuls in the past with csvs using
> data.table::fread(), but readr::read_delim() seems to handle them gracefully
> with just a warning after proceeding.
> Apologies that I do not have a handy reprex. I don't know if I can even
> recreate a parquet file with embedded nuls using arrow if it won't let me
> read one in, and I can't share this file due to company restrictions.
> Please let me know how I can be of any more help!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
