[
https://issues.apache.org/jira/browse/ARROW-13685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Weston Pace updated ARROW-13685:
--------------------------------
Summary: [C++] Cannot write dataset to S3FileSystem if bucket already
exists (was: [Python] Cannot write dataset to S3FileSystem if bucket already
exists)
> [C++] Cannot write dataset to S3FileSystem if bucket already exists
> -------------------------------------------------------------------
>
> Key: ARROW-13685
> URL: https://issues.apache.org/jira/browse/ARROW-13685
> Project: Apache Arrow
> Issue Type: Bug
> Components: Python
> Affects Versions: 5.0.0
> Reporter: Caleb Overman
> Assignee: Weston Pace
> Priority: Major
>
> I'm trying to write a parquet file to an existing S3 bucket using the new
> S3FileSystem interface. However, this is failing with an AWS Access Denied
> error (I do have necessary access). It appears to be trying to recreate the
> bucket which already exists.
> {code:java}
> import numpy as np
> import pyarrow as pa
> from pyarrow import fs
> import pyarrow.dataset as ds
> s3 = fs.S3FileSystem(region="us-west-2")
> table = pa.table({"a": range(10), "b": np.random.randn(10), "c": [1, 2] * 5})
> ds.write_dataset(
> table,
> "my-bucket/test.parquet",
> format="parquet",
> filesystem=s3,
> ){code}
> {code:java}
> OSError: When creating bucket 'my-bucket': AWS Error [code 15]: Access Denied
> {code}
> I'm seeing the same behavior using {{S3FileSystem.create_dir}} when
> {{recursive=True}}.
> {code:java}
> s3.create_dir("my-bucket/test_dir/", recursive=True) # Fails
> s3.create_dir("my-bucket/test_dir/", recursive=False) # Succeeds
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
