[
https://issues.apache.org/jira/browse/ASTERIXDB-3378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ian Maxon resolved ASTERIXDB-3378.
----------------------------------
Resolution: Fixed
> Allow user to opt out of validating write permission before COPY TO S3
> ----------------------------------------------------------------------
>
> Key: ASTERIXDB-3378
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-3378
> Project: Apache AsterixDB
> Issue Type: Task
> Components: EXT - External data
> Affects Versions: 0.9.9
> Reporter: Hussain Towaileb
> Assignee: Hussain Towaileb
> Priority: Major
> Fix For: 0.9.9
>
>
> Currently, for COPY TO S3, one of our early checks validates that:
> * We have the write permission, by writing a small test file to the
> destination, then deleting it, which also tests that
> * Multipart upload is supported, as this is the form of operation we use for
> writing.
>
> This test helps identify failure scenarios early before processing (for
> example) large amount of data, only to fail eventually.
> The limitation of this validation is that it is not always possible to know
> the final destination we will be writing to, depending on the statement
> itself. And so, we always perform this test at the root of the bucket.
> According to the S3 docs, it seems it might be possible to grant the
> permissions only on specific prefixes in a bucket, but not all. This can be
> problematic if we have permission on our destination prefix, but not the root
> of the bucket, because the validation will keep trying to validate against
> the root, which will lead to a failure of the operation.
> This ticket suggests the addition of a flag, e.g.
> {*}"validate-write-permission" (default to true){*}, to skip this operation
> and let the user carry the responsibility of having the right permissions at
> the right time. Note this will only skip the write permission validation, all
> other validations will still take place.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
