James Wing created NIFI-1542:
--------------------------------
Summary: PutS3Object Processor Logs Errors Without
ListBucketMultipartUploads Permission
Key: NIFI-1542
URL: https://issues.apache.org/jira/browse/NIFI-1542
Project: Apache NiFi
Issue Type: Bug
Components: Extensions
Affects Versions: 0.5.0
Environment: Linux, Chrome
Reporter: James Wing
Priority: Minor
In 0.5.0, I am seeing an error putting objects to Amazon S3 using the
PutS3Object processor. The *put operation itself succeeds*, but a nasty-gram
is displayed in the flow and logged as a side effect. The error text is "Error
checking S3 Multipart Upload list for <my s3 bucket>", from line 698 of
PutS3Object.java. This code was introduced in NIFI-1107, expanding the
PutS3Object processor with support for S3 multi-part uploads of larger files.
Logged in nifi-app.log:
{code}
2016-02-19 20:56:24,312 ERROR [Timer-Driven Process Thread-3]
o.a.nifi.processors.aws.s3.PutS3Object PutS
3Object[id=47df7533-91b2-4635-b865-45323f73f6c3] Error checking S3 Multipart
Upload list for batchiq-scra
tch: Access Denied (Service: Amazon S3; Status Code: 403; Error Code:
AccessDenied; Request ID: 7483E76EA
52E33C4)
2016-02-19 20:56:24,533 INFO [Timer-Driven Process Thread-3]
o.a.nifi.processors.aws.s3.PutS3Object PutS3
Object[id=47df7533-91b2-4635-b865-45323f73f6c3] Successfully put
StandardFlowFileRecord[uuid=a4cc19f5-cd2
4-4be7-8157-21f2609fa8cd,claim=StandardContentClaim
[resourceClaim=StandardResourceClaim[id=1455912275117-1, container=default,
section=1], offset=3072, length=1024],offset=0,name=273670477119454,size=1024]
to Amazon S3 in 327 milliseconds
{code}
The error-but-success behavior appears to happen because the exception is
caught, logged, but not re-thrown. The approximate code path is as follows:
# PutS3Object::onTrigger method calls ageoffS3Uploads regardless of
single-part/multi-part code path
# ageoffS3Uploads calls getS3AgeoffListAndAgeoffLocalState
# getS3AgeoffListAndAgeoffLocalState calls s3.listMultipartUploads
# After the exception, onTrigger proceeds with the single-part upload path
*Troubleshooting Notes*
Listing multi-part S3 uploads requires an additional S3 permission,
s3:ListBucketMultipartUploads. I would expect the PutS3Object processor to
only require the s3:PutObject permission for single-part uploads.
I tested that this behavior did not occur in NiFi 0.4.2. I also tested that
adding the s3:ListBucketMultipartUploads permission to my AWS credential
circumvents the error messages from the S3PutObject processor.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
