[
https://issues.apache.org/jira/browse/NIFI-1542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15156231#comment-15156231
]
ASF GitHub Bot commented on NIFI-1542:
--------------------------------------
GitHub user jskora opened a pull request:
https://github.com/apache/nifi/pull/240
NIFI-1542 PutS3Object Processor Logs Errors Without
ListBucketMultipartUploads Permission
Update how PutS3Object handles a 403 AccessDenied response to a S3
ListMultipartUploads request.
* Change log message from error to warning and added note about S3
permissions.
* Advance the AgeOff check time by the check interval as if the request
succeeded to prevent re-checking on every upload when permission does not exist.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/jskora/nifi NIFI-1542
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/nifi/pull/240.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #240
----
commit 4ef45c7d2729b21d9b3fcd6bfc5715360156a3fb
Author: Joe Skora <[email protected]>
Date: 2016-02-21T21:05:12Z
Update how PutS3Object handles a 403 AccessDenied response to a S3
ListMultipartUploads request.
* Change log message from error to warning and added note about S3
permissions.
* Advance the AgeOff check time by the check interval as if the request
succeeded to prevent re-checking on every upload when permission does not exist.
----
> PutS3Object Processor Logs Errors Without ListBucketMultipartUploads
> Permission
> -------------------------------------------------------------------------------
>
> Key: NIFI-1542
> URL: https://issues.apache.org/jira/browse/NIFI-1542
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
> Affects Versions: 0.5.0
> Environment: Linux, Chrome
> Reporter: James Wing
> Priority: Minor
> Fix For: 0.5.1
>
> Attachments: error-checking-s3-multipart-upload-list.png
>
>
> In 0.5.0, I am seeing an error putting objects to Amazon S3 using the
> PutS3Object processor. The *put operation itself succeeds*, but a nasty-gram
> is displayed in the flow and logged as a side effect. The error text is
> "Error checking S3 Multipart Upload list for <my s3 bucket>", from line 698
> of PutS3Object.java. This code was introduced in NIFI-1107, expanding the
> PutS3Object processor with support for S3 multi-part uploads of larger files.
> Logged in nifi-app.log:
> {code}
> 2016-02-19 20:56:24,312 ERROR [Timer-Driven Process Thread-3]
> o.a.nifi.processors.aws.s3.PutS3Object PutS
> 3Object[id=47df7533-91b2-4635-b865-45323f73f6c3] Error checking S3 Multipart
> Upload list for batchiq-scra
> tch: Access Denied (Service: Amazon S3; Status Code: 403; Error Code:
> AccessDenied; Request ID: 7483E76EA
> 52E33C4)
> 2016-02-19 20:56:24,533 INFO [Timer-Driven Process Thread-3]
> o.a.nifi.processors.aws.s3.PutS3Object PutS3
> Object[id=47df7533-91b2-4635-b865-45323f73f6c3] Successfully put
> StandardFlowFileRecord[uuid=a4cc19f5-cd2
> 4-4be7-8157-21f2609fa8cd,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1455912275117-1, container=default,
> section=1], offset=3072,
> length=1024],offset=0,name=273670477119454,size=1024] to Amazon S3 in 327
> milliseconds
> {code}
> The error-but-success behavior appears to happen because the exception is
> caught, logged, but not re-thrown. The approximate code path is as follows:
> # PutS3Object::onTrigger method calls ageoffS3Uploads regardless of
> single-part/multi-part code path
> # ageoffS3Uploads calls getS3AgeoffListAndAgeoffLocalState
> # getS3AgeoffListAndAgeoffLocalState calls s3.listMultipartUploads
> # After the exception, onTrigger proceeds with the single-part upload path
> *Troubleshooting Notes*
> Listing multi-part S3 uploads requires an additional S3 permission,
> s3:ListBucketMultipartUploads. I would expect the PutS3Object processor to
> only require the s3:PutObject permission for single-part uploads.
> I tested that this behavior did not occur in NiFi 0.4.2. I also tested that
> adding the s3:ListBucketMultipartUploads permission to my AWS credential
> circumvents the error messages from the S3PutObject processor.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
