[
https://issues.apache.org/jira/browse/FLINK-24431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Cranmer updated FLINK-24431:
----------------------------------
Description:
*Background*
The EFO Kinesis connector will register and de-register stream consumers based
on the configured [registration
strategy|https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/connectors/datastream/kinesis/#efo-stream-consumer-registrationderegistration].
When {{EAGER}} is used, the client (usually job manager) will register the
consumer and then the task managers will de-register the consumer when job
stops/fails. If the job is configured to restart on fail, then the consumer
will not exist and the job will continuously fail over.
*Solution*
The proposal is to [not deregister the stream
consumer|https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-kinesis/src/main/java/org/apache/flink/streaming/connectors/kinesis/util/StreamConsumerRegistrarUtil.java#L88]
when {{EAGER}} is used. The documentation should be updated to reflect this.
was:
*Background*
The EFO Kinesis connector will register and de-register stream consumers based
on the configured [registration
strategy|https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/connectors/datastream/kinesis/#efo-stream-consumer-registrationderegistration].
When {{EAGER}} is used, the client (usually job manager) will register the
consumer and then the task managers will de-register the consumer when job
stops/fails. If the job is configured to restart on fail, then the consumer
will not exist and the job will continuously fail over.
*Solution*
The proposal is to not deregister the stream consumer when {{EAGER}} is used.
The documentation should be updated to reflect this.
> [Kinesis][EFO] EAGER registration strategy does not work when job fails over
> ----------------------------------------------------------------------------
>
> Key: FLINK-24431
> URL: https://issues.apache.org/jira/browse/FLINK-24431
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Kinesis
> Affects Versions: 1.14.0, 1.12.5, 1.13.2
> Reporter: Danny Cranmer
> Priority: Major
> Fix For: 1.12.6, 1.13.3, 1.15.0, 1.14.1
>
>
> *Background*
> The EFO Kinesis connector will register and de-register stream consumers
> based on the configured [registration
> strategy|https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/connectors/datastream/kinesis/#efo-stream-consumer-registrationderegistration].
> When {{EAGER}} is used, the client (usually job manager) will register the
> consumer and then the task managers will de-register the consumer when job
> stops/fails. If the job is configured to restart on fail, then the consumer
> will not exist and the job will continuously fail over.
> *Solution*
> The proposal is to [not deregister the stream
> consumer|https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-kinesis/src/main/java/org/apache/flink/streaming/connectors/kinesis/util/StreamConsumerRegistrarUtil.java#L88]
> when {{EAGER}} is used. The documentation should be updated to reflect this.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
