Thanks for all the advice. I just thought that 1000 partitions are not that
big, but it turns out it is not :)
Best,
On Sun, Oct 27, 2019 at 6:37 AM Todd Palino wrote:
> Going to pile on here, and also say take that blog post with a grain of
> salt :)
>
> Look at your data size, and your desir
Going to pile on here, and also say take that blog post with a grain of
salt :)
Look at your data size, and your desired degree of parallelism for
consumers, and guide to that. Yes, 1000 partitions takes a while to create.
But unless your data size is something like 1 TB per day or more, you
proba
On Saturday, October 26, 2019, M. Manna wrote:
> You should also check out Becket Qin’s presentation on producer performance
> tuning on YouTube. Both these items should give you all positives and
> negatives of having many/less portions.
>
> Thanks,
>
> On Sat, 26 Oct 2019 at 09:19, Manasvi Gupt
You should also check out Becket Qin’s presentation on producer performance
tuning on YouTube. Both these items should give you all positives and
negatives of having many/less portions.
Thanks,
On Sat, 26 Oct 2019 at 09:19, Manasvi Gupta wrote:
>
> https://www.confluent.io/blog/how-choose-numbe
https://www.confluent.io/blog/how-choose-number-topics-partitions-kafka-cluster
This might be useful for you.
On Sat 26 Oct, 2019, 12:40 PM Jungyeol Lee,
wrote:
> I'm running a kafka cluster running only one broker with GCP n1-standard-2
> instance.
> I configured the number of partitions to 10
I'm running a kafka cluster running only one broker with GCP n1-standard-2
instance.
I configured the number of partitions to 1000, but it is suffering from
running a broker on startup. It seems that it needs much more time for
topic creation. So, what I did is just set the smaller number of partit
