Florian Jetter created ARROW-10146: -------------------------------------- Summary: [Python] Parquet metadata to_dict raises attribute error Key: ARROW-10146 URL: https://issues.apache.org/jira/browse/ARROW-10146 Project: Apache Arrow Issue Type: Bug Components: Python Reporter: Florian Jetter
h2. Description When accessing rowgroup metadata and trying to convert it to a dict using the method {{to_dict}} I sometimes receive an Attribute error. This can be consistently produced with an empty dataframe (see example below) but I have also seen it already for non empty dataframes. I couldn't track down what makes the non-empty Attribute errors special, therefore the example below. h2. Expected behaviour I would expect the to_dict to always, consistently return a dictionary with the appropriate metadata and statistics irregardless of the file content. h2. Minimal Example {code:python} import pandas as pd import pyarrow as pa import pyarrow.parquet as pq df = pd.DataFrame({"col": [1]}).head(0) table = pa.Table.from_pandas(df) buf = pa.BufferOutputStream() pq.write_table(table, buf) reader = pa.BufferReader(buf.getvalue()) parquet_file = pq.ParquetFile(reader) # Raises Attribute Error parquet_file.metadata.to_dict() {code} h3. Traceback {code:java} ~/miniconda3/envs/kartothek-dev/lib/python3.7/site-packages/pyarrow/_parquet.pyx in pyarrow._parquet.FileMetaData.to_dict() ~/miniconda3/envs/kartothek-dev/lib/python3.7/site-packages/pyarrow/_parquet.pyx in pyarrow._parquet.RowGroupMetaData.to_dict() ~/miniconda3/envs/kartothek-dev/lib/python3.7/site-packages/pyarrow/_parquet.pyx in pyarrow._parquet.ColumnChunkMetaData.to_dict() AttributeError: 'NoneType' object has no attribute 'to_dict' {code} h3. Versions {code:java} In [28]: pa.__version__ Out[28]: '1.0.1' In [29]: pd.__version__ Out[29]: '1.0.5' {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)