[
https://issues.apache.org/jira/browse/NIFI-13487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17862992#comment-17862992
]
ASF subversion and git services commented on NIFI-13487:
--------------------------------------------------------
Commit c73d1cf634bbcd384153e3432712eb2afe3dd093 in nifi's branch
refs/heads/main from Krisztina Zsihovszki
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=c73d1cf634 ]
NIFI-13487 ConsumeKinesisStream initial stream position handling error after
AWS SDK 2.x migration
Signed-off-by: Joe Gresock <[email protected]>
This closes #9035.
> ConsumeKinesisStream initial stream position handling error after AWS SDK 2.x
> migration
> ---------------------------------------------------------------------------------------
>
> Key: NIFI-13487
> URL: https://issues.apache.org/jira/browse/NIFI-13487
> Project: Apache NiFi
> Issue Type: Bug
> Affects Versions: 2.0.0-M4
> Reporter: Zsihovszki Krisztina
> Assignee: Zsihovszki Krisztina
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Noticed a changed behaviour for ConsumeKinesisStream processor when
> "Initial Stream Position" is set to "TRIM_HORIZON".
> ConsumeKinesisStream consumes only records which were inserted to Kinesis
> stream after the processor start and it does not consume the records which
> were insterted before processor start.
> In case of "TRIM_HORIZON" the records inserted before processor start (app
> registration) should be consumed as well.
>
> The processor behaved as it used "LATEST" initial stream position.
> The issue occurs for NiFi 2.0 only and the root cause seems to be related to
> NIFI-8531: Upgrading Kinesis processors to AWS SDK 2.x
>
> According to
> [https://docs.aws.amazon.com/streams/latest/dev/kcl-migration.html,] setting
> initialPositionInStreamExtended was moved from LeaseManagementConfig to
> RetrievalConfig.
> The [existing
> implementation|https://github.com/apache/nifi/blob/main/nifi-extension-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/kinesis/stream/ConsumeKinesisStream.java]
> sets the value still in LeaseManagementConfig, this is the reason why the
> default value, "LATEST" was used as initial position.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
