[
https://issues.apache.org/jira/browse/NIFI-1107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15131554#comment-15131554
]
ASF subversion and git services commented on NIFI-1107:
-------------------------------------------------------
Commit 769b044e301d6487a3adbe240df35a92a3e7eb14 in nifi's branch
refs/heads/master from [~jskora]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=769b044 ]
NIFI-1107: Re-integrate Multipart Upload changes into PutS3Object. 1. add
Multipart upload logic to allow resuming an upload after process/instance
restart, 2. add local state management to track the part uploaded for a
flowfile, 3. add configurable AWS S3 state management to abort orphaned
uploads, and 4. adapt to IT test naming.
Pull request updates: * Cleanup property description to reflect processor
details, not code specifics. * Only resume an upload from local state if
uploadID still exists in S3, otherwise delete local state and start a new
upload. * Check that local state from was actually created before trying to
delete it.
Re-integrate Multipart Upload changes into PutS3Object.
1. add Multipart upload logic to allow resuming an upload after
process/instance restart,
2. add local state management to track the part uploaded for a flowfile,
3. add configurable AWS S3 state management to abort orphaned uploads, and
4. adapt to IT test naming.
Fixed test issues
* added mock client for testing local state without matching S3 state,
* updated regex for multipart etag pattern.
Reviewed by Tony Kurc ([email protected]). This closes #192
> Create new PutS3ObjectMultipart processor
> -----------------------------------------
>
> Key: NIFI-1107
> URL: https://issues.apache.org/jira/browse/NIFI-1107
> Project: Apache NiFi
> Issue Type: New Feature
> Components: Extensions
> Reporter: Joe Skora
> Assignee: Joe Skora
> Labels: s3
> Fix For: 0.5.0
>
> Attachments: NIFI-1107.000.patch
>
>
> A new `PutS3ObjectMultipart` processor using the AWS S3 API to upload files
> larger than those supported by `PutS3Object` which has a [5GB
> limit|http://docs.aws.amazon.com/AmazonS3/latest/dev/UploadingObjects.html]
> limit.
> To support S3 compatible endpoints this will also add an `Endpoint Override
> URL` property to `AbstractAWSProcessor` to set the service
> [endpoint|http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/AmazonWebServiceClient.html#setEndpoint(java.lang.String)]
> to override the endpoint URL normally selected based on the the Amazon
> region.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
