[
https://issues.apache.org/jira/browse/ARROW-11793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joris Van den Bossche updated ARROW-11793:
------------------------------------------
Component/s: Parquet
C++
> dictinct_count broken (always 0) in column chunk metadata statistics
> --------------------------------------------------------------------
>
> Key: ARROW-11793
> URL: https://issues.apache.org/jira/browse/ARROW-11793
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++, Parquet, Python
> Affects Versions: 3.0.0
> Reporter: ARF
> Priority: Minor
>
> The {{distinct_count}} attribute of the column chunk metadata statistics is
> broken: It always shows 0. This seems to be the case for all types of
> columns. Checked with int64 as well as dictionary encoded string columns:
> {code:python}
> import pyarrow as pa
> import pyarrow.parquet as pq
> table = pa.Table.from_pydict({
> 'foo': pa.array(['ABC', 'DEF']).dictionary_encode()
> })
> pq.write_table(table, 'test_row_group_statistics.parquet', version='2.0',
> data_page_version='2.0')
> pq_file = pq.ParquetFile('test_row_group_statistics.parquet')
> print(pq_file.metadata.row_group(0).column(0).statistics)
> {code}
> Output:
> {code:none}
> <pyarrow._parquet.Statistics object at 0x0000020A1699D770>
> has_min_max: True
> min: ABC
> max: DEF
> null_count: 0
> distinct_count: 0
> num_values: 2
> physical_type: BYTE_ARRAY
> logical_type: String
> converted_type (legacy): UTF8
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
