[
https://issues.apache.org/jira/browse/ARROW-7156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anthony Abate updated ARROW-7156:
---------------------------------
Description:
I have a 30 gig arrow file with 100 batches. the largest batch in the file
causes get batch to fail - All other batches load fine. in 14.11 the individual
batch errors.. in 15.1.1 the batch crashes R studio when it is used
*14.1.1*
{code:java}
> rbn <- data_rbfr$get_batch(x)
Error in ipc__RecordBatchFileReader_ReadRecordBatch(self, i) :
Invalid: negative malloc size
{code}
*15.1.1*
{code:java}
rbn <- data_rbfr$get_batch(x) works!
df <- as.data.frame(rbn) - Crashes R Studio! {code}
was:
I have a 30 gig arrow file with 100 batches. the largest batch in the file
causes get batch to fail - All other batches load fine. in 14.11 the individual
batch errors.. in 15.1.1 the batch crashes R studio when it is used
*14.1.1*
{code:java}
> rbn <- data_rbfr$get_batch(x)
Error in ipc__RecordBatchFileReader_ReadRecordBatch(self, i) :
Invalid: negative malloc size
{code}
*15.1.1***
{code:java}
rbn <- data_rbfr$get_batch(x) works!
df <- as.data.frame(rbn) - Crashes R Studio! {code}
> [R] [C++] Large Batches Cause Error / Crashes
> ---------------------------------------------
>
> Key: ARROW-7156
> URL: https://issues.apache.org/jira/browse/ARROW-7156
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++, R
> Affects Versions: 0.14.1, 0.15.1
> Reporter: Anthony Abate
> Priority: Major
>
> I have a 30 gig arrow file with 100 batches. the largest batch in the file
> causes get batch to fail - All other batches load fine. in 14.11 the
> individual batch errors.. in 15.1.1 the batch crashes R studio when it is used
> *14.1.1*
> {code:java}
> > rbn <- data_rbfr$get_batch(x)
> Error in ipc__RecordBatchFileReader_ReadRecordBatch(self, i) :
> Invalid: negative malloc size
> {code}
> *15.1.1*
> {code:java}
> rbn <- data_rbfr$get_batch(x) works!
> df <- as.data.frame(rbn) - Crashes R Studio! {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
