0) To: users@kafka.apache.org Cc:
> ja...@scholz.cz Subject: Re: Kafka JVM heap limit
> In addition, in my experience, a memory heap > 8 GB leads to long GC pauses
> which causes the ISR statuses to constantly change, leading to an unstable
> cluster.
>
> --John
>
> On Wed, No
.cz Subject: Re: Kafka JVM heap limit
> In addition, in my experience, a memory heap > 8 GB leads to long GC pauses
> which causes the ISR statuses to constantly change, leading to an unstable
> cluster.
>
> --John
>
> On Wed, Nov 8, 2017 at 4:30 AM, chidigam . wrote:
>
>
Did you use G1GC ?
Thanks
Original message From: John Yost Date:
11/8/17 5:48 AM (GMT-08:00) To: users@kafka.apache.org Cc: ja...@scholz.cz
Subject: Re: Kafka JVM heap limit
In addition, in my experience, a memory heap > 8 GB leads to long GC pauses
which causes the
Hi there,
There are some cases you may want to consider bigger heaps. Heroku runs
some clusters with 20GB heaps now, because said clusters:
1. Use SSL exclusively for connectivity. SSL means message bytes traverse
the JVM heap during encryption/etc, which uses more memory
2. Have a high number of
In addition, in my experience, a memory heap > 8 GB leads to long GC pauses
which causes the ISR statuses to constantly change, leading to an unstable
cluster.
--John
On Wed, Nov 8, 2017 at 4:30 AM, chidigam . wrote:
> Meaning, already read the doc, but couldn't relate, having large Heap for
>
Meaning, already read the doc, but couldn't relate, having large Heap for
JVM will not help
Now it all make sense.
Many thanks.
Bhanu
On Wed, Nov 8, 2017 at 2:34 PM, chidigam . wrote:
> Hi Jakub,
> Thank you very much. I have read this concept I/O cache in design section.
> But couldn't connect
Hi Jakub,
Thank you very much. I have read this concept I/O cache in design section.
But couldn't connect the dots.
Regards
Bhanu
On Wed, Nov 8, 2017 at 2:10 PM, Jakub Scholz wrote:
> This is probably because Kafka uses quite heavily the disk cache maintained
> by the operating system instead o
This is probably because Kafka uses quite heavily the disk cache maintained
by the operating system instead of storing messages in the JVM memory. So
the requirements for the heap memory can be fairly small. The design
section of the documentation describes the details:
http://kafka.apache.org/docu
Hi All,
I have basic question on Kafka JVM configuration, in most of forums I have
seen max heap as 8GB. Why it is not recommended beyond that. Is there any
design limitation ?
Any help in this regards is highly appreciated.
Regards
Bhanu
