compatibility: 0.8.1.1 broker, 0.8.2.2 producer

2015-12-23 Thread Shlomi Hazan
Hi All, Does someone has experience / encountered any issues using a 0.8.2.2 producer against a 0.8.1.1 broker (specifically kafka_2.9.2-0.8.1.1)? I want to upgrade my existing producer (0.8.2-beta). Also, is there a functional difference between the scala versions (2.9.2,2.10,2.11)? Thanks,

Protocol version upgrades in 0.9

2015-12-23 Thread Oleksiy Krivoshey
Hi, I can't understand how the protocol upgrades (to newer version) should work. When I send GroupJoinRequest with a list of assignment protocols (same protocol name, different versions) always the first protocol/version gets picked up as a member version. Even if all consumers in the group are

Re: compatibility: 0.8.1.1 broker, 0.8.2.2 producer

2015-12-23 Thread Andrey Yegorov
I am using 0.8.2.2 producer with 0.8.1.1 brokers without problems. Version of scala matters if you are building with scala or some other components that use scala. Hope this helps. -- Andrey Yegorov On Wed, Dec 23, 2015 at 1:11 PM, Ewen Cheslack-Postava wrote: >

The JIRA Awakens [KAFKA-1841]

2015-12-23 Thread Dana Powers
Hi all, I've been helping debug an issue filed against kafka-python related to compatibility w/ Hortonworks 2.3.0.0 kafka release. As I understand it, HDP is currently based on snapshots of apache/kafka trunk, merged with some custom patches from HDP itself. In this case, HDP's 2.3.0.0 kafka

Gradle build error

2015-12-23 Thread Oliver Pačut
Hello, I am having trouble using Gradle to build Kafka. I get the error: FAILURE: Build failed with an exception. *Where: Build file '/home/user/Desktop/kafka/build.gradle' line: 163 * What went wrong: A problem occurred evaluating root project 'kafka'. > Could not find property 'ScalaPlugin'

Re: NPE on 0.9.0.0 Consumer during broker restart

2015-12-23 Thread Gwen Shapira
Yes, this looks like a bug. Please file a JIRA :) On Wed, Dec 23, 2015 at 1:08 AM, Enrico Olivelli - Diennea < enrico.olive...@diennea.com> wrote: > Hi, > I'm running a brand new Kafka cluster (version 0.9.0.0). During my tests I > noticed this error at Consumer.partitionsFor during a full

Re: Protocol version upgrades in 0.9

2015-12-23 Thread Ewen Cheslack-Postava
Oleksiy, Where are you specifying the version? Unless I'm missing something, the JoinGroup protocol doesn't include versions so I'm not sure I understand the examples you are giving. Are the version numbers included in the per-protocol metadata? You can see exactly how the consumer coordinator

Re: NPE on 0.9.0.0 Consumer during broker restart

2015-12-23 Thread Guozhang Wang
Is this the same issue as KAFKA-2880 ? It is solved in trunk and will be included in 0.9.0.1 release. On Wed, Dec 23, 2015 at 1:14 PM, Gwen Shapira wrote: > Yes, this looks like a bug. Please file a JIRA :) > > On Wed, Dec 23,

Re: compatibility: 0.8.1.1 broker, 0.8.2.2 producer

2015-12-23 Thread Shlomi Hazan
Thank you both Ewen & Andrey! The general rule of upgrading brokers is clear, but it was important for me to hear what other people experienced. Ewen, I assume the broker upgrade from 0.8.1.1 to 0.8.2.2 is as safe as it could be, right? Like I can just take down a single broker, replace jars, and

Re: [kafka-clients] The JIRA Awakens [KAFKA-1841]

2015-12-23 Thread Ewen Cheslack-Postava
Dana, Not sure about the old merge script, but with the new one used for GitHub PRs it tracks the branches you choose to commit to and can update the JIRA automatically, tagging the appropriate fix versions. So from now on, the appropriate JIRA queries for issues with, e.g., fix version 0.9.0.1

Re: Gradle build error

2015-12-23 Thread Ewen Cheslack-Postava
What version of Gradle are you using and can you give the exact command you're running? -Ewen On Wed, Dec 23, 2015 at 5:49 PM, Oliver Pačut wrote: > Hello, > > I am having trouble using Gradle to build Kafka. I get the error: > > > FAILURE: Build failed with an

Re: compatibility: 0.8.1.1 broker, 0.8.2.2 producer

2015-12-23 Thread Ewen Cheslack-Postava
Yes, that should be safe. You can always find notes about upgrades here http://kafka.apache.org/documentation.html#upgrade where it will discuss anything you need to be careful of. -Ewen On Wed, Dec 23, 2015 at 10:51 PM, Shlomi Hazan wrote: > Thank you both Ewen & Andrey! > >

R: NPE on 0.9.0.0 Consumer during broker restart

2015-12-23 Thread Enrico Olivelli - Diennea
Sorry I just created this. I'm going to mark it as duplicate https://issues.apache.org/jira/browse/KAFKA-3041 -Messaggio originale- Da: Guozhang Wang [mailto:wangg...@gmail.com] Inviato: mercoledì 23 dicembre 2015 22:24 A: users@kafka.apache.org Oggetto: Re: NPE on 0.9.0.0 Consumer

Re: Protocol version upgrades in 0.9

2015-12-23 Thread Oleksiy Krivoshey
Hi Ewen, I specify version in ProtocolMetadata structure, as per this document: https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol#AGuideToTheKafkaProtocol-JoinGroupResponse --- ProtocolType => "consumer" ProtocolName => AssignmentStrategy

NPE on 0.9.0.0 Consumer during broker restart

2015-12-23 Thread Enrico Olivelli - Diennea
Hi, I'm running a brand new Kafka cluster (version 0.9.0.0). During my tests I noticed this error at Consumer.partitionsFor during a full cluster restart. My DEV cluster is made of 4 brokers Maybe I can workaround the error doing sanity checks on cluster /using my platform middleware tools)