[
https://issues.apache.org/jira/browse/ARROW-13685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17425706#comment-17425706
]
Weston Pace commented on ARROW-13685:
-------------------------------------
[~1mvdb] I expect you would've been able to get this error with the old path
too. I didn't remove the old behavior or any checks, I just added an
additional check. It might be best to create a new JIRA. It's not clear to me
that translating error code 100 into "bucket already exists" is the correct
thing to do. We could maybe create an auto-retry if we see "Please try again"
but from some googling it appears this condition could take 15 minutes or more
to resolve in some situations (e.g. when creating a bucket with the same name
as a recently deleted bucket).
> [C++] Cannot write dataset to S3FileSystem if bucket already exists
> -------------------------------------------------------------------
>
> Key: ARROW-13685
> URL: https://issues.apache.org/jira/browse/ARROW-13685
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
> Affects Versions: 5.0.0
> Reporter: Caleb Overman
> Assignee: Weston Pace
> Priority: Major
> Labels: pull-request-available
> Fix For: 6.0.0
>
> Time Spent: 4h 50m
> Remaining Estimate: 0h
>
> I'm trying to write a parquet file to an existing S3 bucket using the new
> S3FileSystem interface. However, this is failing with an AWS Access Denied
> error (I do have necessary access). It appears to be trying to recreate the
> bucket which already exists.
> {code:java}
> import numpy as np
> import pyarrow as pa
> from pyarrow import fs
> import pyarrow.dataset as ds
> s3 = fs.S3FileSystem(region="us-west-2")
> table = pa.table({"a": range(10), "b": np.random.randn(10), "c": [1, 2] * 5})
> ds.write_dataset(
> table,
> "my-bucket/test.parquet",
> format="parquet",
> filesystem=s3,
> ){code}
> {code:java}
> OSError: When creating bucket 'my-bucket': AWS Error [code 15]: Access Denied
> {code}
> I'm seeing the same behavior using {{S3FileSystem.create_dir}} when
> {{recursive=True}}.
> {code:java}
> s3.create_dir("my-bucket/test_dir/", recursive=True) # Fails
> s3.create_dir("my-bucket/test_dir/", recursive=False) # Succeeds
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
