[
https://issues.apache.org/jira/browse/FLINK-3079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15026934#comment-15026934
]
Robert Metzger commented on FLINK-3079:
---------------------------------------
Cool. I didn't know this one.
Kafka itself also provides a little tool:
https://cwiki.apache.org/confluence/display/KAFKA/System+Tools#SystemTools-ConsumerOffsetChecker
The purpose of the utility I'm proposing here is to find out the max possible
ingestion rate from Kafka. I'm helping a user on SO who's complaining that
Flink is not reading data fast enough from Kafka
(http://stackoverflow.com/questions/33904793/kafka-flink-performance-issues/33920088#33920088)
, I'm pretty sure its not the Kafka Consumer's fault. To easily prove that,
such a read job would help ;)
> Add utility for measuring the raw read throughput from a Kafka topic
> --------------------------------------------------------------------
>
> Key: FLINK-3079
> URL: https://issues.apache.org/jira/browse/FLINK-3079
> Project: Flink
> Issue Type: New Feature
> Components: Kafka Connector
> Reporter: Robert Metzger
> Assignee: Robert Metzger
> Priority: Minor
>
> I would like to add a little example job to the kafka connector package which
> is measuring the raw read throughput from a kafka topic.
> This is helpful when trying to debug the performance when reading from Kafka.
> Such a tool would help finding the bottleneck in a data processing pipeline.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
