[ https://issues.apache.org/jira/browse/KAFKA-14968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hendrik Jan van Randen reassigned KAFKA-14968: ---------------------------------------------- Assignee: Hendrik Jan van Randen > Consumer Notification Callback > ------------------------------ > > Key: KAFKA-14968 > URL: https://issues.apache.org/jira/browse/KAFKA-14968 > Project: Kafka > Issue Type: New Feature > Components: consumer > Reporter: Hendrik Jan van Randen > Assignee: Hendrik Jan van Randen > Priority: Major > > We want to use Kafka for all the communication between our services. > We have many high frequency topics (many messages per second), for which the > Kafka mechanism of polling by consumers is efficient and suitable. > However, we also have low frequency topics (no or few messages per day). If > such a low frequency message is produced, all consumers should consume it > immediately. But it would be inefficient if all those consumers are polling > continuously with a high frequency (e.g. each second), while no or few such > messages are produced each day. Most efficient would be if for such a low > frequency topic the consumers can register a callback (e.g. a REST endpoint) > with Kafka, and Kafka calls that callback when a message is produced. Either > supplying the message in the call (pro: consumer can handle it immediately), > or just to notify the consumer that it should do a one time poll (pro: same > message transfer mechanism as everywhere). > I just talked this over with Ryan Corrigan of Confluent. > In the coming weeks I will supply more information about the urgency of this > issue. > I will be present at the Kafka Summit in London on May 16 & 17, 2023. Who of > the Confluent technical staff, or Kafka committers, is present at the Kafka > Summit to investigate possible solution directions for this? -- This message was sent by Atlassian Jira (v8.20.10#820010)