Joris Van den Bossche created ARROW-10134:
---------------------------------------------
Summary: [C++][Dataset] Add ParquetFileFragment::num_row_groups
property
Key: ARROW-10134
URL: https://issues.apache.org/jira/browse/ARROW-10134
Project: Apache Arrow
Issue Type: Improvement
Components: C++
Reporter: Joris Van den Bossche
Fix For: 2.0.0
>From https://github.com/dask/dask/pull/6534#issuecomment-699512602, comment by
>[~rjzamora]:
bq. it would be great to have access the total row-group count for the
fragment from a {{num_row_groups}} attribute (which pyarrow should be able to
get without parsing all row-group metadata/statistics - I think?).
One question is: does this attribute correspond to the row groups in the
parquet file, or the (subset of) row groups represented by the fragment?
I expect the second (so if you do SplitByRowGroup, you would get a fragment
with num_row_groups==1), but this might be a potential confusing aspect of the
attribute.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
