[jira] [Updated] (ARROW-3410) [C++] Streaming CSV reader interface for memory-constrainted environments
[ https://issues.apache.org/jira/browse/ARROW-3410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wes McKinney updated ARROW-3410: Labels: dataset (was: ) > [C++] Streaming CSV reader interface for memory-constrainted environments > - > > Key: ARROW-3410 > URL: https://issues.apache.org/jira/browse/ARROW-3410 > Project: Apache Arrow > Issue Type: New Feature > Components: C++ >Reporter: Wes McKinney >Priority: Major > Labels: dataset > > CSV reads are currently all-or-nothing. If the results of parsing a CSV file > do not fit into memory, this can be a problem. I propose to define a > streaming {{RecordBatchReader}} interface so that the record batches produced > by reading can be written out immediately to a stream on disk, to be memory > mapped later -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (ARROW-3410) [C++] Streaming CSV reader interface for memory-constrainted environments
[ https://issues.apache.org/jira/browse/ARROW-3410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wes McKinney updated ARROW-3410: Fix Version/s: (was: 0.14.0) > [C++] Streaming CSV reader interface for memory-constrainted environments > - > > Key: ARROW-3410 > URL: https://issues.apache.org/jira/browse/ARROW-3410 > Project: Apache Arrow > Issue Type: New Feature > Components: C++ >Reporter: Wes McKinney >Priority: Major > > CSV reads are currently all-or-nothing. If the results of parsing a CSV file > do not fit into memory, this can be a problem. I propose to define a > streaming {{RecordBatchReader}} interface so that the record batches produced > by reading can be written out immediately to a stream on disk, to be memory > mapped later -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-3410) [C++] Streaming CSV reader interface for memory-constrainted environments
[ https://issues.apache.org/jira/browse/ARROW-3410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wes McKinney updated ARROW-3410: Fix Version/s: (was: 0.12.0) 0.13.0 > [C++] Streaming CSV reader interface for memory-constrainted environments > - > > Key: ARROW-3410 > URL: https://issues.apache.org/jira/browse/ARROW-3410 > Project: Apache Arrow > Issue Type: New Feature > Components: C++ >Reporter: Wes McKinney >Priority: Major > Fix For: 0.13.0 > > > CSV reads are currently all-or-nothing. If the results of parsing a CSV file > do not fit into memory, this can be a problem. I propose to define a > streaming {{RecordBatchReader}} interface so that the record batches produced > by reading can be written out immediately to a stream on disk, to be memory > mapped later -- This message was sent by Atlassian JIRA (v7.6.3#76005)