[
https://issues.apache.org/jira/browse/ARROW-17046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amir Khosroshahi updated ARROW-17046:
-------------------------------------
Description:
According to PyArrow 8.0.0
[documentation|https://arrow.apache.org/docs/python/generated/pyarrow.parquet.write_to_dataset.html]
{{kwargs}} is "Additional kwargs for {{write_table}} function." However when I
try to pass additional arguments, for example {{{}flavor{}}}, to the underlying
write_table I get the following error
{code:java}
TypeError: unexpected parquet write option: flavor{code}
This used to work in PyArrow as late as 7.0.0 but started to break in 8.0.0.
Minimal example to reproduce the error
{code:java}
import pyarrow as pa
import pandas as pd
import pyarrow.parquet as pq
df = pd.DataFrame({'a': [1, 2], 'b': [3, 4]})
tb = pa.Table.from_pandas(df)
pq.write_to_dataset(tb, "test.parquet", flavor="spark") {code}
was:
According to PyArrow 8.0.0
[documentation|https://arrow.apache.org/docs/python/generated/pyarrow.parquet.write_to_dataset.html]
{{kwargs}} is "Additional kwargs for {{write_table}} function." However when I
try to pass for example flavor as an additional argument to the underlying
write_table I get the following error
{code:java}
TypeError: unexpected parquet write option: flavor{code}
This used to work in PyArrow as late as 7.0.0 but started to break in 8.0.0.
Minimal example to reproduce the error
{code:java}
import pyarrow as pa
import pandas as pd
import pyarrow.parquet as pq
df = pd.DataFrame({'a': [1, 2], 'b': [3, 4]})
tb = pa.Table.from_pandas(df)
pq.write_to_dataset(tb, "test.parquet", flavor="spark") {code}
> [Python] pyarrow.parquet.write_to_dataset fails to pass kwargs to write_table
> function
> --------------------------------------------------------------------------------------
>
> Key: ARROW-17046
> URL: https://issues.apache.org/jira/browse/ARROW-17046
> Project: Apache Arrow
> Issue Type: Bug
> Components: Python
> Affects Versions: 8.0.0
> Reporter: Amir Khosroshahi
> Priority: Minor
>
> According to PyArrow 8.0.0
> [documentation|https://arrow.apache.org/docs/python/generated/pyarrow.parquet.write_to_dataset.html]
> {{kwargs}} is "Additional kwargs for {{write_table}} function." However when
> I try to pass additional arguments, for example {{{}flavor{}}}, to the
> underlying write_table I get the following error
>
> {code:java}
> TypeError: unexpected parquet write option: flavor{code}
>
> This used to work in PyArrow as late as 7.0.0 but started to break in 8.0.0.
> Minimal example to reproduce the error
>
> {code:java}
> import pyarrow as pa
> import pandas as pd
> import pyarrow.parquet as pq
> df = pd.DataFrame({'a': [1, 2], 'b': [3, 4]})
> tb = pa.Table.from_pandas(df)
> pq.write_to_dataset(tb, "test.parquet", flavor="spark") {code}
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
