[
https://issues.apache.org/jira/browse/NIFI-1542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15155830#comment-15155830
]
Tony Kurc commented on NIFI-1542:
---------------------------------
It is possible, but I can describe the intent behind where it is. Our concern
was that if your upload pattern is few multiparts, mixed with mostly small
puts, the ageoff would not occur often enough. I can think of a few fixes:
1. move the call as you suggested
2. switch the log to warn, and live with making extra calls
3. adding an option to never try ageoff on the processor
4. consider a failed multipart list due to permissions a "successful" attempt
to ageoff, which should make the calls happen very infrequently
[~jskora], [~jameswing] thoughts?
> PutS3Object Processor Logs Errors Without ListBucketMultipartUploads
> Permission
> -------------------------------------------------------------------------------
>
> Key: NIFI-1542
> URL: https://issues.apache.org/jira/browse/NIFI-1542
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
> Affects Versions: 0.5.0
> Environment: Linux, Chrome
> Reporter: James Wing
> Priority: Minor
> Fix For: 0.5.1
>
> Attachments: error-checking-s3-multipart-upload-list.png
>
>
> In 0.5.0, I am seeing an error putting objects to Amazon S3 using the
> PutS3Object processor. The *put operation itself succeeds*, but a nasty-gram
> is displayed in the flow and logged as a side effect. The error text is
> "Error checking S3 Multipart Upload list for <my s3 bucket>", from line 698
> of PutS3Object.java. This code was introduced in NIFI-1107, expanding the
> PutS3Object processor with support for S3 multi-part uploads of larger files.
> Logged in nifi-app.log:
> {code}
> 2016-02-19 20:56:24,312 ERROR [Timer-Driven Process Thread-3]
> o.a.nifi.processors.aws.s3.PutS3Object PutS
> 3Object[id=47df7533-91b2-4635-b865-45323f73f6c3] Error checking S3 Multipart
> Upload list for batchiq-scra
> tch: Access Denied (Service: Amazon S3; Status Code: 403; Error Code:
> AccessDenied; Request ID: 7483E76EA
> 52E33C4)
> 2016-02-19 20:56:24,533 INFO [Timer-Driven Process Thread-3]
> o.a.nifi.processors.aws.s3.PutS3Object PutS3
> Object[id=47df7533-91b2-4635-b865-45323f73f6c3] Successfully put
> StandardFlowFileRecord[uuid=a4cc19f5-cd2
> 4-4be7-8157-21f2609fa8cd,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1455912275117-1, container=default,
> section=1], offset=3072,
> length=1024],offset=0,name=273670477119454,size=1024] to Amazon S3 in 327
> milliseconds
> {code}
> The error-but-success behavior appears to happen because the exception is
> caught, logged, but not re-thrown. The approximate code path is as follows:
> # PutS3Object::onTrigger method calls ageoffS3Uploads regardless of
> single-part/multi-part code path
> # ageoffS3Uploads calls getS3AgeoffListAndAgeoffLocalState
> # getS3AgeoffListAndAgeoffLocalState calls s3.listMultipartUploads
> # After the exception, onTrigger proceeds with the single-part upload path
> *Troubleshooting Notes*
> Listing multi-part S3 uploads requires an additional S3 permission,
> s3:ListBucketMultipartUploads. I would expect the PutS3Object processor to
> only require the s3:PutObject permission for single-part uploads.
> I tested that this behavior did not occur in NiFi 0.4.2. I also tested that
> adding the s3:ListBucketMultipartUploads permission to my AWS credential
> circumvents the error messages from the S3PutObject processor.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
