Re: Loggly's use of Kafka on AWS

2013-12-03 Thread Timothy Chen
Hi Philip, So I wonder if you guys hit disk perf problems with EBS? It seems quite common in the past but I haven't tried recently. Also can you share how you guys deployed zookeeper in AWS so that a qurom is always available? Tim Sent from my iPhone On Dec 2, 2013, at 5:15 PM, Steve Morin

Re: Loggly's use of Kafka on AWS

2013-12-03 Thread Philip O'Toole
Yeah, we use provisioned IOPs EBS so we have good performance. We run 3 ZK nodes, one in each AZ. Quorum 2. Philip On Dec 3, 2013, at 7:31 AM, Timothy Chen tnac...@gmail.com wrote: Hi Philip, So I wonder if you guys hit disk perf problems with EBS? It seems quite common in the past but

set new retention size on the fly

2013-12-03 Thread Yu, Libo
Hi folks, For 0.8, it is possible to add a partition dynamically. Is it possible to increase the retention size on the fly? This feature will be very useful for operation. I know rolling start can pick up the change but it takes too much effort. Thanks. Libo

Re: set new retention size on the fly

2013-12-03 Thread Neha Narkhede
For adding partitions, look at https://cwiki.apache.org/confluence/display/KAFKA/Replication+tools#Replicationtools-5.AddPartitionTool 0.8.1 includes the ability to dynamically change per topic configs. Thanks, Neha On Tue, Dec 3, 2013 at 10:21 AM, Yu, Libo libo...@citi.com wrote: Hi folks,

RE: set new retention size on the fly

2013-12-03 Thread Yu, Libo
Hi Neha, 0.8.1 includes the ability to dynamically change per topic configs. Do you mean number of partitions or retention size? Regards, Libo -Original Message- From: Neha Narkhede [mailto:neha.narkh...@gmail.com] Sent: Tuesday, December 03, 2013 1:25 PM To: users@kafka.apache.org

snappy 1.0.4.1, java7, macos issue...

2013-12-03 Thread Jason Rosenberg
So, It looks like there is an issue with using snappy version 1.0.4.1, with java 7, on MacOSX, which is fixed in version 1.0.5. We've verified this. https://github.com/ptaoussanis/carmine/issues/5 Perhaps 1.0.5 should be the default for kafka 0.8 (or maybe too late for that?). I've filed:

Re: Using Kafka 0.8 from Scala and Akka

2013-12-03 Thread Richard Rodseth
Any update on this, Chetan? Thanks. On Thu, Oct 31, 2013 at 4:11 PM, chetan conikee coni...@gmail.com wrote: I am in the process of releasing out Scala and RxJava consumer(s) on github. Will be releasing it soon. Keep an eye out. On Thu, Oct 31, 2013 at 3:49 PM, Richard Rodseth