ASF GitHub Bot commented on NIFI-1769:

Github user miquillo commented on the issue:

    ouch, that's painful... we've just committed very similar code in NIFI-1769 
(PR #362, see reference), since we required a kinesis streams processor block. 
    I propose to take this solution further though, as it seems more mature and 
already has been reviewed a few times. 
    My comments:
    - At the time of reading, the name of the issue / PR didn't make us realize 
enough that this processor block was already build. Kinesis is the Umbrella, 
which currently contains three services: Firehose, Streams and Analytics. This 
processor block only works for Kinesis Streams, Please rename both code and 
PR/Jira :) (+ also the tests under a subfolder 'streams')
    - The API explains some restrictions to PutRecords(): 
    > Each PutRecords request can support up to 500 records. Each record in the 
request can be as large as 1 MB, up to a limit of 5 MB for the entire request, 
including partition keys. Each shard can support writes up to 1,000 records per 
second, up to a maximum data write total of 1 MB per 
    Data needs to be chunked into multiple PutRecords calls if the amount of 
records > 500. We used the following:
    `List<PutRecordBatchResponseEntry> recordChuncks = Lists.partition(records, 
    - We added a NR_SHARDS parameter for the sake of resharding (scaling 
up/down) in the future. Although we didn't implement a resharding mechanism, 
it's perhaps worth considering.

> Add support for SSE-KMS and S3 Signature Version 4 Authentication AWS
> ---------------------------------------------------------------------
>                 Key: NIFI-1769
>                 URL: https://issues.apache.org/jira/browse/NIFI-1769
>             Project: Apache NiFi
>          Issue Type: Improvement
>          Components: Core Framework
>    Affects Versions: 0.5.1
>            Reporter: Michiel Moonen
>            Priority: Minor
>              Labels: newbie, patch, security
> Currently there is no support for SSE-KMS S3 Signature Version 4 
> Authentication. This is necessary for enhanced security features

This message was sent by Atlassian JIRA

Reply via email to