Max0u opened a new issue, #37389:
URL: https://github.com/apache/arrow/issues/37389
### Describe the bug, including details regarding any error messages,
version, and platform.
Hello,
While reading a generated parquet file from AWS Athena (Trino engine behind)
we have the following error:
```
File
"pypoetry/virtualenvs/lib/python3.10/site-packages/pyarrow/parquet/core.py",
line 308, in __init__
self.reader.open(
File "pyarrow/_parquet.pyx", line 1239, in
pyarrow._parquet.ParquetReader.open
File "pyarrow/error.pxi", line 100, in pyarrow.lib.check_status
pyarrow.lib.ArrowInvalid: Map keys must be annotated as required.
```
(parquet-tools give the same error if you want to reproduce)
We don't have this error will reading it with spark. Is it Spark that's
being lax with parquet spec? Or an issue with Arrow?
Athena Query if you want to reproduce
```sql
UNLOAD (
SELECT ARRAY [
MAP(ARRAY['key_1', 'key_2'], ARRAY['value_1', 'value_2']),
MAP(ARRAY['key_1', 'key_2'], ARRAY['value_2', 'value_2']),
MAP(ARRAY['key_1', 'key_2'], ARRAY['value_3', 'value_2'])
] AS array_map
)
TO 'path'
WITH (format = 'PARQUET')
```
If you want directly the file:
https://file.io/Bs6BtISt3GFb
### Component(s)
Parquet, Python
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
