Follower brokers not syncing with 0.8.2

2015-02-06 Thread Albert Strasheim
Hello folks We have currently have a cluster of 20 nodes running Kafka 0.8.1.1 serving a large volume of production traffic. I'm bringing up another 20 nodes with Kafka 0.8.2.0 in the same cluster, currently serving a test topic with 32 partitions spread across the 20 new nodes, with replication

Re: question about new consumer offset management in 0.8.2

2015-02-06 Thread Joel Koshy
On Thu, Feb 05, 2015 at 11:57:15PM -0800, Joel Koshy wrote: > On Fri, Feb 06, 2015 at 12:43:37AM -0500, Jason Rosenberg wrote: > > I'm not sure what you mean by 'default' behavior 'only if' offset.storage > > is kafka. Does that mean the 'default' behavior is 'false' if > > offset.storage is 'zoo

Re: java API equivalent of --from-beginning?

2015-02-06 Thread Yang
ah... thanks. I was looking at the latest github code (commit f1ba4ff87e60a6dc06fb3d16e84fa94f8c4a7d9e ) and it doesn't show that line 181 I'll try it out On Fri, Feb 6, 2015 at 5:37 AM, Joe Stein wrote: > You have to delete the znode from zookeeper for your consumer group > > https://github.co

Re: high level consumer shutdown exception

2015-02-06 Thread Jarek Jarcec Cecho
Obavam se ze jsem jediny dalsi clovek na tehle skupine kdo rozumi cesky, takze bych doporucil psat dotazy v anglictine :) (I believe that I’m the only other guy who can speak Czech on this group, so It might be good idea to write questions in english) Jarcec > On Feb 6, 2015, at 4:42 AM, zden

Re: Kafka broker core dump

2015-02-06 Thread Gwen Shapira
Hi Xinyi, The core dump never made it to the list. Perhaps its too large for an attachment. Can you upload it somewhere and send a link? Also, the broker logs will be helpful, and a description of what the test did. Gwen On Thu, Feb 5, 2015 at 11:27 PM, Xinyi Su wrote: > Hi, > > I encounter a

high level consumer shutdown exception

2015-02-06 Thread Zdenek . Tison
Pokud není v této zprávě výslovně uvedeno jinak, platí, že: - žádné skutečnosti a údaje v této zprávě obsažené nelze považovat za závazný návrh na uzavření smlouvy nebo přijetí Vašeho návrhu na uzavření smlouvy. - obsahuje-li tato zpráva návrh na uzavření smlouvy (nabídku), pak jakékoli i nepods

Re: java API equivalent of --from-beginning?

2015-02-06 Thread Joe Stein
You have to delete the znode from zookeeper for your consumer group https://github.com/apache/kafka/blob/0.8.1/core/src/main/scala/kafka/consumer/ConsoleConsumer.scala#L181 /*** Joe Stein Founder, Principal Consultant Big Data Open Source Security LLC ht

java API equivalent of --from-beginning?

2015-02-06 Thread Yang
we are trying to achive the equivalent of bin/kafka-console-consumer.sh we tried " props.put("auto.offset.reset", "smallest"); but it still doesn't work. how can I achieve --from-beginning in java code? thanks (we are using 8.1.1)

Re: question about new consumer offset management in 0.8.2

2015-02-06 Thread Joel Koshy
On Fri, Feb 06, 2015 at 12:43:37AM -0500, Jason Rosenberg wrote: > I'm not sure what you mean by 'default' behavior 'only if' offset.storage > is kafka. Does that mean the 'default' behavior is 'false' if > offset.storage is 'zookeeper'? Can that be clarified in the config > documentation section