[
https://issues.apache.org/jira/browse/FLINK-27399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated FLINK-27399:
-----------------------------------
Labels: pull-request-available (was: )
> Pulsar connector didn't set start consuming position correctly
> --------------------------------------------------------------
>
> Key: FLINK-27399
> URL: https://issues.apache.org/jira/browse/FLINK-27399
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Pulsar
> Affects Versions: 1.15.0, 1.14.4, 1.16.0
> Reporter: Yufan Sheng
> Assignee: Yufan Sheng
> Priority: Major
> Labels: pull-request-available
>
> The Pulsar connector didn't use the consuming position from the checkpoint.
> They just commit the position to Pulsar after the checkpoint is complete. And
> the connector start to consume message from Pulsar directly by the offset
> stored on the Pulsar subscription.
> This causes the test could be failed in some situations. The start cursor
> (position on Pulsar) would be reset to the wrong position which causes the
> results didn't match the desired records.
> This issue fixes
> # FLINK-23944
> # FLINK-25884
> # FLINK-26177
> # FLINK-26237
> # FLINK-26721
> Although the test failure message could be various. They are truly sharing
> the same cause.
> h2. How to fix this issue:
> SourceEvent protocol for limiting the {{Consumer.seek}} operation.
> The Pulsar source needs to seek the desired consuming position when
> bootstrapping. The seeking action couldn’t be executed concurrently. We have
> designed a [new
> mechanism|https://github.com/apache/flink/pull/17119#pullrequestreview-746035072].
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
