[
https://issues.apache.org/jira/browse/ARROW-3774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16719705#comment-16719705
]
Wes McKinney commented on ARROW-3774:
-------------------------------------
The main use case would be for pandas (where things need to be contiguous), but
there memory will have to be copied in general when calling
{{pyarrow.Table.to_pandas}}, so the benefits of this optimization would be
minimal, if any. Producing large contiguous arrays could even be more expensive
than the current behavior of creating chunked arrays
> [C++] Change parquet::arrow::FileReader::ReadRowGroups to read into
> contiguous arrays
> -------------------------------------------------------------------------------------
>
> Key: ARROW-3774
> URL: https://issues.apache.org/jira/browse/ARROW-3774
> Project: Apache Arrow
> Issue Type: Improvement
> Components: C++
> Reporter: Uwe L. Korn
> Assignee: Wes McKinney
> Priority: Major
> Labels: parquet
>
> Instead of creating a chunk per RowGroup, we should read at least for
> primitive type into a single, pre-allocated Array. This needs some new
> functionality in the Record reader classes and thus should be done after
> https://github.com/apache/parquet-cpp/pull/462 is merged.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
