[
https://issues.apache.org/jira/browse/ARROW-453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16907590#comment-16907590
]
Wes McKinney edited comment on ARROW-453 at 8/14/19 8:00 PM:
-------------------------------------------------------------
[~pitrou] one consideration is that data might be spread across multiple
buckets with the same credentials, am I understanding that right? As some other
data points, TensorFlow's implementation seems to treat the bucket as part of
the file path
https://github.com/tensorflow/tensorflow/blob/master/tensorflow/core/platform/s3/s3_file_system_test.cc
Turi Create (formerly Dato Create) accepts fully qualified URLs including even
AWS keys, bucket name, and object name
https://github.com/apple/turicreate/blob/master/src/core/storage/fileio/s3_api.hpp#L177
was (Author: wesmckinn):
What does the "fsspec model" mean here? The Python API you mean?
[~pitrou] one consideration is that data might be spread across multiple
buckets with the same credentials, am I understanding that right? As some other
data points, TensorFlow's implementation seems to treat the bucket as part of
the file path
https://github.com/tensorflow/tensorflow/blob/master/tensorflow/core/platform/s3/s3_file_system_test.cc
Turi Create (formerly Dato Create) accepts fully qualified URLs including even
AWS keys, bucket name, and object name
https://github.com/apple/turicreate/blob/master/src/core/storage/fileio/s3_api.hpp#L177
> [C++] Add file interface implementations for Amazon S3
> ------------------------------------------------------
>
> Key: ARROW-453
> URL: https://issues.apache.org/jira/browse/ARROW-453
> Project: Apache Arrow
> Issue Type: New Feature
> Components: C++
> Reporter: Wes McKinney
> Priority: Major
> Labels: filesystem
> Fix For: 1.0.0
>
>
> The BSD-licensed C++ code in SFrame
> (https://github.com/turi-code/SFrame/tree/master/oss_src/fileio) may provide
> some inspiration.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
