[
https://issues.apache.org/jira/browse/NIFI-15586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18058899#comment-18058899
]
ASF subversion and git services commented on NIFI-15586:
--------------------------------------------------------
Commit 86bd16364cc1ae861b790e00c952c823c6f62680 in nifi's branch
refs/heads/main from Alaksiej Ščarbaty
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=86bd16364c ]
NIFI-15586 Add demarcator processing strategy to ConsumeKinesis
This closes #10904.
Signed-off-by: Pierre Villard <[email protected]>
> Add DEMARCATOR strategy to ConsumeKinesis
> -----------------------------------------
>
> Key: NIFI-15586
> URL: https://issues.apache.org/jira/browse/NIFI-15586
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
> Reporter: Alaksiej Ščarbaty
> Assignee: Alaksiej Ščarbaty
> Priority: Major
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Currently ConsumeKinesis supports FLOW_FILE and RECORD processing strategies.
> When writing FlowFiles RECORD strategy groups Kinesis records by the source
> shard and record schema. In situations where every record has its own unique
> schema, every record is written into a separate FlowFile.
> With DEMARCATOR strategy (like in ConsumeKafka) it will be possible to write
> all records from a single shard into a single FlowFile no matter the schema.
> This will significantly improve performance for cases where records with
> different schemas are present in a Kinesis stream.
> Changes:
> Add DEMARCATOR option to Processing Strategy property.
> Add MESSAGE_DEMARCATOR property, which will be used to separate Kinesis
> records. The property is available only when DEMARCATOR is selected.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
