[
https://issues.apache.org/jira/browse/ARROW-1231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17055656#comment-17055656
]
Frank Natividad commented on ARROW-1231:
----------------------------------------
The XML API does exist and compatibility with S3 SDK is available but not for
all operations. Not exhaustive, but three things I think about when telling you
this:
* Permission management with IAM isn't available through the XML API and S3
SDK would need to rely on ACL management. The GCS team aligned with GCP
permission management by supporting IAM policies at the
Organization/Project/Bucket level. IAM policies is the canonical way to manage
permissions on GCP. For example, Uniform Bucket Level Access was introduced to
disable ACLs at Bucket and Object level and is required to support Conditional
permission policies (IAM Conditions). ACL are still supported but not the
canonical permission management and very specific to GCS.
* S3 API isn't fully supported through the XML API, for example, S3 multi-part
uploads are not supported. Moreover, you'd need to develop around these
limitations when you hit them.
* The GCS C++ library was design based on experience from user friction from
existing libraries in other languages in mind.
In this case, I would recommend the GCS C++ library if you're using C++ and
Cloud Storage API over the S3 SDK.
HTH
> [C++] Add filesystem / IO implementation for Google Cloud Storage
> -----------------------------------------------------------------
>
> Key: ARROW-1231
> URL: https://issues.apache.org/jira/browse/ARROW-1231
> Project: Apache Arrow
> Issue Type: New Feature
> Components: C++
> Reporter: Wes McKinney
> Priority: Major
> Labels: filesystem
>
> See example jumping off point
> https://github.com/tensorflow/tensorflow/tree/master/tensorflow/core/platform/cloud
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
