[
https://issues.apache.org/jira/browse/ARROW-7825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17033841#comment-17033841
]
Neal Richardson edited comment on ARROW-7825 at 2/10/20 6:34 PM:
-----------------------------------------------------------------
[~khughitt] Try writing iris to feather and parquet with Species as a character
column and reading it back. My guess is that it will stay a character/string
column and also will ignore {{stringsAsFactors}}.
Leaving aside the merits of the {{stringsAsFactors}} feature itself and its
historical origins, I think it only really makes sense when you're reading a
text file like a CSV and you don't have type information available. For binary
formats with metadata, it makes sense to preserve what was saved. When you
{{readRDS}} it doesn't convert your character vectors to factors, for example:
{code}
> options(stringsAsFactors=FALSE)
> tf <- tempfile()
> saveRDS(iris, tf)
> str(readRDS(tf))
'data.frame': 150 obs. of 5 variables:
$ Sepal.Length: num 5.1 4.9 4.7 4.6 5 5.4 4.6 5 4.4 4.9 ...
$ Sepal.Width : num 3.5 3 3.2 3.1 3.6 3.9 3.4 3.4 2.9 3.1 ...
$ Petal.Length: num 1.4 1.4 1.3 1.5 1.4 1.7 1.4 1.5 1.4 1.5 ...
$ Petal.Width : num 0.2 0.2 0.2 0.2 0.2 0.4 0.3 0.2 0.2 0.1 ...
$ Species : Factor w/ 3 levels "setosa","versicolor",..: 1 1 1 1 1 1 1 1 1
1 ...
> iris$Species <- as.character(iris$Species)
> str(iris)
'data.frame': 150 obs. of 5 variables:
$ Sepal.Length: num 5.1 4.9 4.7 4.6 5 5.4 4.6 5 4.4 4.9 ...
$ Sepal.Width : num 3.5 3 3.2 3.1 3.6 3.9 3.4 3.4 2.9 3.1 ...
$ Petal.Length: num 1.4 1.4 1.3 1.5 1.4 1.7 1.4 1.5 1.4 1.5 ...
$ Petal.Width : num 0.2 0.2 0.2 0.2 0.2 0.4 0.3 0.2 0.2 0.1 ...
$ Species : chr "setosa" "setosa" "setosa" "setosa" ...
> saveRDS(iris, tf)
> str(readRDS(tf))
'data.frame': 150 obs. of 5 variables:
$ Sepal.Length: num 5.1 4.9 4.7 4.6 5 5.4 4.6 5 4.4 4.9 ...
$ Sepal.Width : num 3.5 3 3.2 3.1 3.6 3.9 3.4 3.4 2.9 3.1 ...
$ Petal.Length: num 1.4 1.4 1.3 1.5 1.4 1.7 1.4 1.5 1.4 1.5 ...
$ Petal.Width : num 0.2 0.2 0.2 0.2 0.2 0.4 0.3 0.2 0.2 0.1 ...
$ Species : chr "setosa" "setosa" "setosa" "setosa" ...
{code}
was (Author: npr):
[~khughitt] Try writing iris to feather and parquet with Species as a character
column and reading it back. My guess is that it will stay a character/string
column and also will ignore {{stringsAsFactors}}.
Leaving aside the merits of the {{stringsAsFactors}} feature itself and its
historical origins, I think it only really makes sense when you're reading a
text file like a CSV and you don't have type information available. For binary
formats with metadata, it makes sense to preserve what was saved. When you
{{readRDS}} it doesn't convert your character vectors to factors, for example:
{code}
> options(stringsAsFactors=FALSE)
> tf <- tempfile()
> saveRDS(iris, tf)
> str(readRDS(tf))
'data.frame': 150 obs. of 5 variables:
$ Sepal.Length: num 5.1 4.9 4.7 4.6 5 5.4 4.6 5 4.4 4.9 ...
$ Sepal.Width : num 3.5 3 3.2 3.1 3.6 3.9 3.4 3.4 2.9 3.1 ...
$ Petal.Length: num 1.4 1.4 1.3 1.5 1.4 1.7 1.4 1.5 1.4 1.5 ...
$ Petal.Width : num 0.2 0.2 0.2 0.2 0.2 0.4 0.3 0.2 0.2 0.1 ...
$ Species : Factor w/ 3 levels "setosa","versicolor",..: 1 1 1 1 1 1 1 1 1
1 ...
{code}
> Have arrow::read_parquet respect options(stringsAsFactors = FALSE)
> ------------------------------------------------------------------
>
> Key: ARROW-7825
> URL: https://issues.apache.org/jira/browse/ARROW-7825
> Project: Apache Arrow
> Issue Type: Improvement
> Components: R
> Affects Versions: 0.16.0
> Environment: Linux 64-bit 5.4.15
> Reporter: Keith Hughitt
> Priority: Major
> Labels: R, parquet
>
> Same issue as reported for feather::read_feather
> (https://issues.apache.org/jira/browse/ARROW-7823);
>
> For the R arrow package, the "read_parquet()" function currently does not
> respect "options(stringsAsFactors = FALSE)", leading to
> unexpected/inconsistent behavior.
>
> *Example:*
>
>
> {code:java}
> library(arrow)
> library(readr)
> options(stringsAsFactors = FALSE)
> write_tsv(head(iris), 'test.tsv')
> write_parquet(head(iris), 'test.parquet')
> head(read.delim('test.tsv', sep='\t')$Species)
> # [1] "setosa" "setosa" "setosa" "setosa" "setosa" "setosa"
> head(read_tsv('test.tsv', col_types = cols())$Species)
> # [1] "setosa" "setosa" "setosa" "setosa" "setosa" "setosa"
> head(read_parquet('test.parquet')$Species)
> # [1] setosa setosa setosa setosa setosa setosa
> # Levels: setosa versicolor virginica
> {code}
>
>
> *Versions:*
> - R 3.6.2
> - arrow_0.15.1.9000
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
