[
https://issues.apache.org/jira/browse/ARROW-17308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alenka Frim updated ARROW-17308:
--------------------------------
Description:
Documentation for PyArrow 6.x and 7.x both indicate that the `validate_schema`
argument is supported in the `ParquetDataset` class. Yet passing that argument
to an instance results in:
ValueError: Keyword 'validate_schema' is not yet supported with the new Dataset
API
Code:
{code:python}
parquet_dataset = pyarrow.parquet.ParquetDataset(
path_or_paths=paths,
validate_schema=validate_schema,
filesystem=filesystem,
partitioning=partitioning,
use_legacy_dataset=False,
){code}
Docs link:
[https://arrow.apache.org/docs/6.0/python/generated/pyarrow.parquet.ParquetDataset.html]
[https://arrow.apache.org/docs/7.0/python/generated/pyarrow.parquet.ParquetDataset.html]
was:
Documentation for PyArrow 6.x and 7.x both indicate that the `validate_schema`
argument is supported in the `ParquetDataset` class. Yet passing that argument
to an instance results in:
ValueError: Keyword 'validate_schema' is not yet supported with the new Dataset
API
Code:
```
parquet_dataset = pyarrow.parquet.ParquetDataset(
path_or_paths=paths,
validate_schema=validate_schema,
filesystem=filesystem,
partitioning=partitioning,
use_legacy_dataset=False,
)
```
Docs link:
[https://arrow.apache.org/docs/6.0/python/generated/pyarrow.parquet.ParquetDataset.html]
[https://arrow.apache.org/docs/7.0/python/generated/pyarrow.parquet.ParquetDataset.html]
> ValueError: Keyword 'validate_schema' is not yet supported with the new
> Dataset API
> -----------------------------------------------------------------------------------
>
> Key: ARROW-17308
> URL: https://issues.apache.org/jira/browse/ARROW-17308
> Project: Apache Arrow
> Issue Type: Bug
> Components: Python
> Reporter: Abderrahmane Jaidi
> Priority: Major
> Labels: dataset-parquet-legacy, dataset-parquet-read
>
> Documentation for PyArrow 6.x and 7.x both indicate that the
> `validate_schema` argument is supported in the `ParquetDataset` class. Yet
> passing that argument to an instance results in:
> ValueError: Keyword 'validate_schema' is not yet supported with the new
> Dataset API
> Code:
> {code:python}
> parquet_dataset = pyarrow.parquet.ParquetDataset(
> path_or_paths=paths,
> validate_schema=validate_schema,
> filesystem=filesystem,
> partitioning=partitioning,
> use_legacy_dataset=False,
> ){code}
> Docs link:
> [https://arrow.apache.org/docs/6.0/python/generated/pyarrow.parquet.ParquetDataset.html]
> [https://arrow.apache.org/docs/7.0/python/generated/pyarrow.parquet.ParquetDataset.html]
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
