[
https://issues.apache.org/jira/browse/NIFI-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15564923#comment-15564923
]
ASF GitHub Bot commented on NIFI-1769:
--------------------------------------
Github user miquillo commented on the issue:
https://github.com/apache/nifi/pull/239
ouch, that's painful... we've just committed very similar code in NIFI-1769
(PR #362, see reference), since we required a kinesis streams processor block.
I propose to take this solution further though, as it seems more mature and
already has been reviewed a few times.
My comments:
- At the time of reading, the name of the issue / PR didn't make us realize
enough that this processor block was already build. Kinesis is the Umbrella,
which currently contains three services: Firehose, Streams and Analytics. This
processor block only works for Kinesis Streams, Please rename both code and
PR/Jira :) (+ also the tests under a subfolder 'streams')
- The API explains some restrictions to PutRecords():
> Each PutRecords request can support up to 500 records. Each record in the
request can be as large as 1 MB, up to a limit of 5 MB for the entire request,
including partition keys. Each shard can support writes up to 1,000 records per
second, up to a maximum data write total of 1 MB per
second.[http://docs.aws.amazon.com/kinesis/latest/APIReference/API_PutRecords.html]
Data needs to be chunked into multiple PutRecords calls if the amount of
records > 500. We used the following:
`List<PutRecordBatchResponseEntry> recordChuncks = Lists.partition(records,
500);`
- We added a NR_SHARDS parameter for the sake of resharding (scaling
up/down) in the future. Although we didn't implement a resharding mechanism,
it's perhaps worth considering.
> Add support for SSE-KMS and S3 Signature Version 4 Authentication AWS
> ---------------------------------------------------------------------
>
> Key: NIFI-1769
> URL: https://issues.apache.org/jira/browse/NIFI-1769
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
> Affects Versions: 0.5.1
> Reporter: Michiel Moonen
> Priority: Minor
> Labels: newbie, patch, security
>
> Currently there is no support for SSE-KMS S3 Signature Version 4
> Authentication. This is necessary for enhanced security features
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
