[
https://issues.apache.org/jira/browse/ARROW-8385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wes McKinney updated ARROW-8385:
--------------------------------
Summary: [Python][Parquet] Crash on parquet.read_table on windows python
3.82 (was: Crash on parquet.read_table on windows python 3.82)
> [Python][Parquet] Crash on parquet.read_table on windows python 3.82
> --------------------------------------------------------------------
>
> Key: ARROW-8385
> URL: https://issues.apache.org/jira/browse/ARROW-8385
> Project: Apache Arrow
> Issue Type: Bug
> Components: Python
> Affects Versions: 0.16.0
> Environment: Window 10
> python 3.8.2 pip 20.0.2
> pip freeze ->
> numpy==1.18.2
> pandas==1.0.3
> pyarrow==0.16.0
> python-dateutil==2.8.1
> pytz==2019.3
> six==1.14.0
> Reporter: Geoff Quested-Joens
> Priority: Major
> Fix For: 0.17.0
>
> Attachments: crash.parquet
>
>
> On read of parquet file using pyarrow the program spontaneously exits no
> thrown exceptions windows only. Testing the same setup on linux (debian 10 in
> a Docker) reading the same parquet file is done without issue.
> The follow can reproduce the crash in a python 3.8.2 environment env listed
> bellow but is essentially pip install pandas and pyarrow.
> {code:python}
> import pandas as pd
> import pyarrow as pa
> import pyarrow.parquet as pq
> def test_pandas_write_read():
> df_out = pd.DataFrame.from_dict([{"A":i} for i in range(3)])
> df_out.to_parquet("crash.parquet")
> df_in = pd.read_parquet("crash.parquet")
> print(df_in)
> def test_arrow_write_read():
> df = pd.DataFrame.from_dict([{"A":i} for i in range(3)])
> table_out = pa.Table.from_pandas(df)
> pq.write_table(table_out, 'crash.parquet')
> table_in = pq.read_table('crash.parquet')
> print(table_in)
> if _name_ == "_main_":
> test_pandas_write_read()
> test_arrow_write_read()
> {code}
> The interpreter never reaches the print statements crashing somewhere in the
> call on line 252 of {{parquet.py}} no error is thrown just spontaneous
> program exit.
> {code:python}
> self.reader.read_all(...
> {code}
> In contrast running the same code and python environment in debian 10 there
> is no error reading the parquet files generated by the same windows code. The
> sha2sum compare equal for the crash.parquet generated running on debian and
> windows so something appears to be up with the read. Attached is the
> crash.parquet file generated on my machine.
> Obtusely changing the {{range(3)}} to {{range(2)}} gets rid of the crash on
> windows.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
