[
https://issues.apache.org/jira/browse/KAFKA-6020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17782515#comment-17782515
]
Murari Goswami edited comment on KAFKA-6020 at 11/3/23 10:55 AM:
-----------------------------------------------------------------
We are also suffered with moving expansive amount of event data at consumer
side and then throw away the majority of the records as one of the consumer is
interested in specific data. A broker side filter will help a lot in reducing
network load in moving these unwanted data. However, in order to achieve that
deserialization has to be done at broker level. This will inherently increase
the load on the broker. So, while we like to have this feature but this could
bring other bottlenecks. A detail architectural evaluation needed.
was (Author: JIRAUSER302514):
We are also suffered with moving expansive amount of event data at consumer
side and then throw away the majority of the records as one of the consumer is
interested in specific data. A broker side filter will help a lot in reducing
network load in moving these unwanted data. So if this can be a part of feature
in kafka in next release will be of great help.
> Broker side filtering
> ---------------------
>
> Key: KAFKA-6020
> URL: https://issues.apache.org/jira/browse/KAFKA-6020
> Project: Kafka
> Issue Type: New Feature
> Components: consumer
> Reporter: Pavel Micka
> Priority: Major
> Labels: needs-kip
>
> Currently, it is not possible to filter messages on broker side. Filtering
> messages on broker side is convenient for filter with very low selectivity
> (one message in few thousands). In my case it means to transfer several GB of
> data to consumer, throw it away, take one message and do it again...
> While I understand that filtering by message body is not feasible (for
> performance reasons), I propose to filter just by message key prefix. This
> can be achieved even without any deserialization, as the prefix to be matched
> can be passed as an array (hence the broker would do just array prefix
> compare).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
