[
https://issues.apache.org/jira/browse/ARROW-14930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17456624#comment-17456624
]
Luis Morales edited comment on ARROW-14930 at 12/10/21, 9:11 AM:
-----------------------------------------------------------------
The thing is that Im not using pyarrow to create dir's as in your use case...
in my test I use s3cmd to create folders and move files, in our production use
case I'll use parquet files generated with Apache Spark with Hive partitioning
was (Author: JIRAUSER281000):
The thing is that Im not using pyarrow to create dir's as in your use case and
not scmd to create folders and move files... in our production use case I'll
use parquet files generated with Apache Spark with Hive partitioning, so...
> [C++][Python] FileNotFound with Scality accessed through S3 APIs
> ----------------------------------------------------------------
>
> Key: ARROW-14930
> URL: https://issues.apache.org/jira/browse/ARROW-14930
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++, Python
> Affects Versions: 6.0.1
> Environment: linux + python 3.8
> Reporter: Luis Morales
> Priority: Major
> Labels: s3
> Fix For: 6.0.2
>
>
> When using dataset.Dataset with S3FileSystem with compatible S3 object
> sotrage, get an FileNotFoundError.
>
> My code:
>
> scality = fs.S3FileSystem(access_key='accessKey1',
> secret_key='verySecretKey1', endpoint_override="http://localhost:8000";,
> region="")
> data = ds.dataset("dasynth/parquet/taxies/2019_june/", format="parquet",
> partitioning="hive", filesystem=scality)
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
