[
https://issues.apache.org/jira/browse/ARROW-7740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17070604#comment-17070604
]
Wes McKinney commented on ARROW-7740:
-------------------------------------
I've hit the limit of my knowledge of how the R C API works. See this commit
https://github.com/wesm/arrow/commit/b5f07442c0297b85030b8624faced5bb114bb849#diff-85a7e5bb1bf1ea13536e9956b1920093R589
I placed Validate usages on either side of {{SET_VECTOR_ELT}} here
{code}
auto ingest_one = [&](R_xlen_t i) {
auto slice =
values_array->Slice(list_array->value_offset(i),
list_array->value_length(i));
STOP_IF_NOT_OK(slice->Validate());
STOP_IF_NOT_OK(values_array->Validate());
SET_VECTOR_ELT(data, i + start, Array__as_vector(slice));
STOP_IF_NOT_OK(values_array->Validate());
STOP_IF_NOT_OK(slice->Validate());
};
{code}
The code {{SET_VECTOR_ELT(data, i + start, Array__as_vector(slice));}} causes
memory corruption (because the Validate calls fail on the subsequent lines) so
presumably there is some missing use of Protect or other APIs
> [R] Crash/bad data in converting Arrow list struct type
> -------------------------------------------------------
>
> Key: ARROW-7740
> URL: https://issues.apache.org/jira/browse/ARROW-7740
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++, R
> Reporter: John Sheffield
> Assignee: Wes McKinney
> Priority: Minor
> Fix For: 0.17.0
>
>
> Reading a nested ndjson file using arrow::read_json_arrow with the default
> `as_data_frame=TRUE` causes an immediate session crash, but switching to
> `as_data_frame=FALSE` works fine and the resulting arrow object schema is
> correct.
> {code:java}
> library(tidyr)
> library(arrow)
> library(jsonlite)
> # Create two test datasets: long_df and a variant that nests long_df into
> # a dataframe with a list-column 'nest_level1' containing a dataframe
> long_df <- tidyr::expand_grid(ABC = LETTERS[1:3], xyz = letters[24:26], num =
> 1:3)
> long_df[["ftr1"]] <- runif(nrow(long_df))
> long_df[["ftr2"]] <- rpois(nrow(long_df), 100)
> nested_frame_level1 <- tidyr::nest(long_df, nest_level1 = c(num, ftr1, ftr2))
> # Write and validate nested ndjson
> jsonlite::stream_out(nested_frame_level1, con =
> file("nested_frame_level1.json"))
> readLines("nested_frame_level1.json", n = 2) # check we have valid ndjson here
> # This does not cause a session crash
> nested_arrow <- arrow::read_json_arrow(file = "nested_frame_level1.json",
> as_data_frame = FALSE)
> nested_arrow$schema # correctly interprets 'nest_level1` as `list<item:
> struct<num: int64, ftr1: double, ftr2: int64>>`
> # This causes a session crash
> nested_df <- arrow::read_json_arrow(file = "nested_frame_level1.json",
> as_data_frame = TRUE)
>
> {code}
> The R package version of Arrow is latest CRAN release (arrow * 0.15.1.1,
> 2019-11-05, CRAN (R 3.5.2)). I'm running this code in a slightly older R
> version (3.5.1), macOS 10.14.6, x86_64, darwin15.6.0, via RStudio 1.2.5001.
> [edit: formatting fix]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
