[
https://issues.apache.org/jira/browse/ARROW-16991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ogego updated ARROW-16991:
--------------------------
Description:
As documented here:
[https://github.com/apache/parquet-format/blob/master/PageIndex.md]
The parquet format supports indexing of pages within a column chunk. Beyond
what is accomplished by rowgroup statistics, this can further accelerate
scanning.
As far as I am able to tell, Arrow does not currently support these indices (I
see no references to {{{}offset_index_offset{}}}, {{{}column_index_offset{}}},
etc). Are there plans to add support, either in C++ or pyarrow?
was:
As documented here:
[https://github.com/apache/parquet-format/blob/master/PageIndex.md]
The parquet format supports indexing of pages within a column chunk. Beyond
what is accomplished by rowgroup statistics, this can further accelerate
scanning.
As far as I am able to tell, Arrow does not currently support these indices (I
see no references to `offset_index_offset`, `column_index_offset`, etc). Are
there plans to add support, either in C++ or pyarrow?
> [C++][Parquet][Python] Add Parquet ColumnIndex and OffsetIndex support
> ----------------------------------------------------------------------
>
> Key: ARROW-16991
> URL: https://issues.apache.org/jira/browse/ARROW-16991
> Project: Apache Arrow
> Issue Type: New Feature
> Components: C++, Parquet, Python
> Reporter: ogego
> Priority: Minor
>
> As documented here:
> [https://github.com/apache/parquet-format/blob/master/PageIndex.md]
> The parquet format supports indexing of pages within a column chunk. Beyond
> what is accomplished by rowgroup statistics, this can further accelerate
> scanning.
> As far as I am able to tell, Arrow does not currently support these indices
> (I see no references to {{{}offset_index_offset{}}},
> {{{}column_index_offset{}}}, etc). Are there plans to add support, either in
> C++ or pyarrow?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
