Re: On the number of partitions.

2019-10-26 Thread Jungyeol Lee
Thanks for all the advice. I just thought that 1000 partitions are not that big, but it turns out it is not :) Best, On Sun, Oct 27, 2019 at 6:37 AM Todd Palino wrote: > Going to pile on here, and also say take that blog post with a grain of > salt :) > > Look at your data size, and your desir

Re: Kafka 2.2.1 with OpenJDK 11

2019-10-26 Thread Debraj Manna
Thanks Jonathan. On Sun 27 Oct, 2019, 12:13 AM Jonathan Santilli, wrote: > Hello, > > I can confirm that the version 2.2.0 supports OpenJDK 8 in the broker side, > we are using it on production. > > > Cheers! > -- > Jonathan > > On Sat, Oct 26, 2019, 4:47 PM Debraj Manna > wrote: > > > Kafka la

Re: On the number of partitions.

2019-10-26 Thread Todd Palino
Going to pile on here, and also say take that blog post with a grain of salt :) Look at your data size, and your desired degree of parallelism for consumers, and guide to that. Yes, 1000 partitions takes a while to create. But unless your data size is something like 1 TB per day or more, you proba

Re: On the number of partitions.

2019-10-26 Thread Edward Capriolo
On Saturday, October 26, 2019, M. Manna wrote: > You should also check out Becket Qin’s presentation on producer performance > tuning on YouTube. Both these items should give you all positives and > negatives of having many/less portions. > > Thanks, > > On Sat, 26 Oct 2019 at 09:19, Manasvi Gupt

Re: Kafka 2.2.1 with OpenJDK 11

2019-10-26 Thread Jonathan Santilli
Hello, I can confirm that the version 2.2.0 supports OpenJDK 8 in the broker side, we are using it on production. Cheers! -- Jonathan On Sat, Oct 26, 2019, 4:47 PM Debraj Manna wrote: > Kafka latest doc mention about JDK 8. But make no mention of openjdk or > Oracle JDK. Can someone confirm i

Re: Kafka 2.2.1 with OpenJDK 11

2019-10-26 Thread Debraj Manna
Kafka latest doc mention about JDK 8. But make no mention of openjdk or Oracle JDK. Can someone confirm if JDK 8 means openjdk 8 is also supported? https://kafka.apache.org/documentation/#java On Fri 25 Oct, 2019, 9:17 PM Debraj Manna, wrote: > Thanks for the pointers. > > Does kafka 2.2.1 supp

Re: Broker that stays outside of the ISR, how to recover

2019-10-26 Thread Bart van Deenen
Hi all Thanks for the help. Eventually restarting the broker a second time (days later) triggered a full repair, and the cluster is happy again. I don't know why the first restart didn't fix the issue. Greetings Bart On Fri, Oct 18, 2019, at 16:09, M. Manna wrote: > In addition to what Peter

Re: On the number of partitions.

2019-10-26 Thread M. Manna
You should also check out Becket Qin’s presentation on producer performance tuning on YouTube. Both these items should give you all positives and negatives of having many/less portions. Thanks, On Sat, 26 Oct 2019 at 09:19, Manasvi Gupta wrote: > > https://www.confluent.io/blog/how-choose-numbe

Re: On the number of partitions.

2019-10-26 Thread Manasvi Gupta
https://www.confluent.io/blog/how-choose-number-topics-partitions-kafka-cluster This might be useful for you. On Sat 26 Oct, 2019, 12:40 PM Jungyeol Lee, wrote: > I'm running a kafka cluster running only one broker with GCP n1-standard-2 > instance. > I configured the number of partitions to 10

On the number of partitions.

2019-10-26 Thread Jungyeol Lee
I'm running a kafka cluster running only one broker with GCP n1-standard-2 instance. I configured the number of partitions to 1000, but it is suffering from running a broker on startup. It seems that it needs much more time for topic creation. So, what I did is just set the smaller number of partit