[
https://issues.apache.org/jira/browse/FLINK-38287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18016306#comment-18016306
]
Shekhar Prasad Rajak commented on FLINK-38287:
----------------------------------------------
I have some design in mind and will be sharing the draft design and approach.
Meantime feel free to share your thoughts & ideas for the project.
> Kafka 4.1.0 Queue Semantics support in Flink Connector Kafka
> ------------------------------------------------------------
>
> Key: FLINK-38287
> URL: https://issues.apache.org/jira/browse/FLINK-38287
> Project: Flink
> Issue Type: New Feature
> Components: API / Core
> Affects Versions: kafka-4.1.0, flink-2.0.0
> Reporter: Shekhar Prasad Rajak
> Priority: Major
>
> KIP-932 introduces share groups as a new consumption model that provides
> queue semantics.
> This directly addresses *use cases* where:
> 1. Multiple consumers need to process items efficiently in parallel from a
> single/multiple topic(s).
> 2. Messages need explicit acknowledgment/release (to avoid reprocessing or
> allow retries).
> h4. *Operational Benefits*
> - {*}Higher Throughput{*}: ShareGroupHeartbeat helps in Queue-like
> workloads, maximum throughput scenarios. Share groups distribute messages at
> the record level, not partition level, so multiple readers can consume from
> the same topic with Kafka coordinating message distribution.
> - {*}Better Availability and Flexible Scaling{*}: consumers assignment
> logic is simpler in server side and rebalancing frequency is minimised.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
