Alaksiej Ščarbaty created NIFI-15586:
----------------------------------------
Summary: Add DEMARCATOR strategy to ConsumeKinesis
Key: NIFI-15586
URL: https://issues.apache.org/jira/browse/NIFI-15586
Project: Apache NiFi
Issue Type: Improvement
Reporter: Alaksiej Ščarbaty
Assignee: Alaksiej Ščarbaty
Currently ConsumeKinesis supports FLOW_FILE and RECORD processing strategies.
When writing FlowFiles RECORD strategy groups Kinesis records by the source
shard and record schema. In situations where every record has its own unique
schema, every record is written into a separate FlowFile.
With DEMARCATOR strategy (like in ConsumeKafka) it will be possible to write
all records from a single shard into a single FlowFile no matter the schema.
This will significantly improve performance for cases where records with
different schemas are present in a Kinesis stream.
Changes:
Add DEMARCATOR option to Processing Strategy property.
Add MESSAGE_DEMARCATOR property, which will be used to separate Kinesis
records. The property is available only when DEMARCATOR is selected.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
