[
https://issues.apache.org/jira/browse/KAFKA-16835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17850425#comment-17850425
]
Andrew Schofield commented on KAFKA-16835:
------------------------------------------
This is a very significant change to how Kafka works. Kafka does actually know
the commit order so it is technically possible to understand the commit order
of records with some processing. But.......
Because records are added to topic-partitions in produce order, not commit
order, and the offsets are assigned when the records are added, commit-order
consumption would not deliver records across transactions in offset order. It
seems very tricky to me to understand how to track consumer position (committed
offsets) when the records are being delivered out of order. The combination of
commit-order delivery and consumer groups seems incompatible with the current
client API behaviour.
In summary, this would be a big piece of work that would likely require deep
changes in the broker as well as the client. Not a small effort.
> Add Support for consumer to read in commit order.
> -------------------------------------------------
>
> Key: KAFKA-16835
> URL: https://issues.apache.org/jira/browse/KAFKA-16835
> Project: Kafka
> Issue Type: New Feature
> Components: clients, consumer, offset manager
> Reporter: Manjunath
> Priority: Critical
>
> Currently consumer supports offset order to receive messages.There are some
> cases where commit order is very important.For example assume case where
> PostgreSQL-14 randomly streams multiple in-progress large transactions to
> some intermediate client which starts transactional producer instances for
> multiple in-progress transactions,using this producer instances client pushes
> data to kafka. Now consumer should strictly read messages based on commit
> order.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
