[
https://issues.apache.org/jira/browse/GEARPUMP-40?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Manu Zhang updated GEARPUMP-40:
-------------------------------
Description:
As per
[https://engineering.linkedin.com/kafka/benchmarking-apache-kafka-2-million-writes-second-three-cheap-machines],
a single thread Kafka consumer could consume at 89.7MB/s from 6x partition 3x
replica topic whose data are evenly distributed on a 3-node GbE cluster. I
carried out a similar experiment with KafkaSource and found that the throughput
is only at 10MB/s.
was:
As per
[https://engineering.linkedin.com/kafka/benchmarking-apache-kafka-2-million-writes-second-three-cheap-machines],
a single thread Kafka consumer could consume at 89.7MB/s from 6x partition 3x
replica topic. I carried out a similar experiment with KafkaSource and found
that the throughput is only at 10MB/s.
> KafkaSource poor performance
> ----------------------------
>
> Key: GEARPUMP-40
> URL: https://issues.apache.org/jira/browse/GEARPUMP-40
> Project: Apache Gearpump
> Issue Type: Improvement
> Components: kafka
> Affects Versions: 0.8.0
> Reporter: Manu Zhang
> Assignee: Manu Zhang
>
> As per
> [https://engineering.linkedin.com/kafka/benchmarking-apache-kafka-2-million-writes-second-three-cheap-machines],
> a single thread Kafka consumer could consume at 89.7MB/s from 6x partition
> 3x replica topic whose data are evenly distributed on a 3-node GbE cluster. I
> carried out a similar experiment with KafkaSource and found that the
> throughput is only at 10MB/s.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
