Re: JAVA HEAP settings for KAFKA in production

Just about all of our systems are actually running 64GB of memory now, and yes, we want plenty of space for cache with all the disk and network IO going on. -Todd On 5/7/14, 10:31 AM, "John Penner" wrote: >Just curious if these JAVA_HEAP tunings are still for machines listed in >6.4 >Hardware a

Re: JAVA HEAP settings for KAFKA in production

Just curious if these JAVA_HEAP tunings are still for machines listed in 6.4 Hardware and OS : "We are using dual quad-core Intel Xeon machines with 24GB of memory." Seems like a small heap size for 24GB of memory, or is that done to maximize as

Re: JAVA HEAP settings for KAFKA in production

Updated the docs with Todd's recommendation - http://kafka.apache.org/081/documentation.html#java Thanks, Neha On Tue, May 6, 2014 at 9:40 AM, Jay Kreps wrote: > Hey Todd, > > Doc patch? :-) > > svn co http://svn.apache.org/repos/asf/kafka/site/081/ > > Don't stress about html or formatting, I

Re: JAVA HEAP settings for KAFKA in production

Hey Todd, Doc patch? :-) svn co http://svn.apache.org/repos/asf/kafka/site/081/ Don't stress about html or formatting, I'm happy to do that part. I would love to give people more authoritative advice. Right now everything is a bit obsolete and wrong. -Jay On Mon, May 5, 2014 at 10:36 PM, Todd

Re: JAVA HEAP settings for KAFKA in production

I apologize for taking a couple days to jump in on this. We¹re currently running JDK 1.7 u51, and we¹ve switched over to the G1 collector. If you do this (and I highly recommend it), make sure you¹re on u51. We tried out u21 in testing, but we had a number of problems with the GC implementation in

Re: JAVA HEAP settings for KAFKA in production

Hi, We have 2 small EC2 instances acting as "Receivers" for SPM. Each of them runs Apache, Jetty, and a Kafka broker with the following params: java -Xmx512M -server -Dlog4j.configuration=file:/usr/lib/kafka/bin/../config/log4j.properties -cp :/usr/lib/kafka/bin/../proj

Re: JAVA HEAP settings for KAFKA in production

The GC settings at http://kafka.apache.org/documentation.html#java are old. We meant to update the documentation with the new GC settings using the G7 collector, but we haven't gotten around to doing that. Let me reach out to our engineer, Todd Palino, who worked on tuning GC for Kafka at LinkedIn

Re: JAVA HEAP settings for KAFKA in production

http://kafka.apache.org/documentation.html#java Thanks, Jun On Thu, May 1, 2014 at 12:19 PM, Cassa L wrote: > Hi, > I want to know what usually, are the JAVA_HEAP settings recommended for > kafka servers in production. > > Thanks > LCassa >

JAVA HEAP settings for KAFKA in production

Hi, I want to know what usually, are the JAVA_HEAP settings recommended for kafka servers in production. Thanks LCassa