post is to
> explain a few important determining factors and provide a few simple
> formulas. More... |
> > | |
> > | View on blog.confluent.io | Preview by Yahoo |
> > | |
> > | |
> >
> > Naidu Saladi
> >
> > From: Daniel Compton <daniel.compton.l
and provide a few simple formulas. More... |
| |
| View on blog.confluent.io | Preview by Yahoo |
| |
| |
Naidu Saladi
From: Daniel Compton daniel.compton.li...@gmail.com
To: users@kafka.apache.org
Sent: Wednesday, May 20, 2015 8:21 PM
Subject: Re: Optimal number of partitions
a few simple
formulas. More... |
| |
| View on blog.confluent.io | Preview by Yahoo |
| |
| |
Naidu Saladi
From: Daniel Compton daniel.compton.li...@gmail.com
To: users@kafka.apache.org
Sent: Wednesday, May 20, 2015 8:21 PM
Subject: Re: Optimal number of partitions
Hi,
We are implementing a Kafka cluster with 9 brokers into EC2 instances, and we
are trying to find out the optimal number of partitions for our topics, finding
out the maximal number in order not to update the partition number anymore.
What we understood is that the number of partitions
Hi,
We are implementing a Kafka cluster with 9 brokers into EC2 instances, and we
are trying to find out the optimal number of partitions for our topics, finding
out the maximal number in order not to update the partition number anymore.
What we understood is that the number of partitions
With knowing the actual implementation details, I would get guess more
partitions implies more parallelism, more concurrency, more threads, more
files to write to - all of which will contribute to more CPU load.
Partitions allow you to scale by partitioning the topic across multiple
brokers.
One of the beautiful things about Kafka is that it uses the disk and OS
disk caching really efficiently.
Because Kafka writes messages to a contiguous log, it needs very little
seek time to move the write head to the next point. Similarly for reading,
if the consumers are mostly up to date with
Subject: Re: Optimal number of partitions for topic
One of the beautiful things about Kafka is that it uses the disk and OS
disk caching really efficiently.
Because Kafka writes messages to a contiguous log, it needs very little
seek time to move the write head to the next point. Similarly
