Re: [DISCUSSION] Kafka 0.8.2.2 release?

2015-08-15 Thread Joe Stein
Is this something also that should be in 0.8.2.2 https://issues.apache.org/jira/browse/KAFKA-2421 wasn't sure what the lz4 usage has been seems to break on some jdks. +1 on the snappy fixes ~ Joe Stein On Fri, Aug 14, 2015 at 5:39 PM, Guozhang Wang wrote: > +1 for both KAFKA-2189 and 2308. > >

Re: use page cache as much as possiblee

2015-08-15 Thread Kishore Senji
Please check this: https://lonesysadmin.net/2013/12/22/better-linux-disk-caching-performance-vm-dirty_ratio/ I depends on your OS, please research those parameters appropriately and see what they are on your current system. Based on those parameters the background syncing will be done by the OS. B

Re: Replica not available...when it is!

2015-08-15 Thread Andrew Otto
Ah likely not. Will check. Do you have a link to the commit? > On Aug 15, 2015, at 13:34, Clark Haskins wrote: > > I know we ran into the same issue with Camus at LinkedIn and it has since > been fixed. I hope that we committed the patch to open source. > > Are you running the latest versi

Re: Replica not available...when it is!

2015-08-15 Thread Clark Haskins
I know we ran into the same issue with Camus at LinkedIn and it has since been fixed. I hope that we committed the patch to open source. Are you running the latest version of Camus? -Clark Sent from my iPhone > On Aug 15, 2015, at 10:25 AM, Andrew Otto wrote: > > Hm, interesting. So my real

Re: Replica not available...when it is!

2015-08-15 Thread Andrew Otto
Hm, interesting. So my real issue is more with Camus than with cluster problems? It seems that Camus won’t consume if it encounters a ReplicaNotAvailableException. > On Aug 15, 2015, at 12:02, Clark Haskins wrote: > > Replica not available is not a fatal exception. This simply means that th

thread that handle client request in Kafka brokers

2015-08-15 Thread Tao Feng
Hi All, I was told that only single thread handle client producer request in Kafka broker which may potentially be a performance problem with request queue-up if we have many small requests. I am wondering which part of the code I should read to understand the above logic Thanks, -Tao

Re: Replica not available...when it is!

2015-08-15 Thread Clark Haskins
Replica not available is not a fatal exception. This simply means that there is a replica that is down. If you get Leader not available that means the partition is offline. -Clark Sent from my iPhone > On Aug 15, 2015, at 8:41 AM, Andrew Otto wrote: > > Also strange: If I start this broker

Re: Replica not available...when it is!

2015-08-15 Thread Andrew Otto
Also strange: If I start this broker back up, and then issue a kafkacat metadata request, I do not see any 'Broker: Replica not available’, even though this broker’s preferred partitions have not yet replicated back in sync, and are not the leader. Everything seems normal. Somehow this broker

Replica not available...when it is!

2015-08-15 Thread Andrew Otto
I am having trouble with a single broker causing consumers to lag. As I am troubleshooting this issue, I have stopped this broker in the hopes that other replicas will take over as leader for this broker’s preferred partitions. However, when I do so, Camus reports: kafka.CamusJob: Skipping