Re: can kafka 10 stream API read the topic from a Kafka 9 cluster?
This is a recognized area for improvement and better version compatibility is something that is being actively worked on. librdkafka clients already allow for both forward and backward compatibility. Soon the java clients will be able to do so as well. -hans > On Dec 24, 2016, at 12:26 AM, Sachin Mittal wrote: > > What I understand is the Kafka has following apis > 1. Core broker APIs which runs the server. (Heart of kafka) > 2. Producer API > 3. Consumer API > 4. Connect API > 5. Streams API > > All four use broker for different purpose. > There should be some sort of compatibility matrix. > > Kafka server is something central to all applications and one where > frequent upgrades are not possible in a production environment. > > So every release of other future APIs should have some backward > compatibility with Kafka server. > > Like x.y.z.w server should atleast be compatible with > x.y.**.** APIs. > > So any new release x.y.**.** should make sure that they are compatible > oldest x.y.**.** release atleast. > > And perhaps some sort of compatibility matrix page should be made available. > > Just my 2 cents. > > Thanks > Sachin > > > > >> On Sat, Dec 24, 2016 at 12:05 AM, Hans Jespersen wrote: >> >> No. All Java clients (including Streams) need to be the same version (or >> lower) as the brokers they connect to. >> >> -hans >> >> >> >> >>> On Dec 23, 2016, at 1:03 AM, Sachin Mittal wrote: >>> >>> Is Kafka streams 0.10.2.0-SNAPSHOT compatible with 0.10.0.1 broker. >>> I was facing broker connect issue and it just did not work. >>> >>> Thanks >>> Sachin >>> >>> On Fri, Dec 23, 2016 at 4:25 AM, wrote: Kafka clients (currently) do not work against older Kafka >> brokers/servers so you have no other option but to upgrade to a 0.10.1.0 or higher Kafka broker. -hans > On Dec 22, 2016, at 2:25 PM, Joanne Contact wrote: > > Hello I have a program which requires 0.10.1.0 streams API. The jar is > packaged by maven with all dependencies. I tried to consume a Kafka > topic spit from a Kafka 9 cluster. > > It has such error: > org.apache.kafka.common.protocol.types.SchemaException: Error reading > field 'topic_metadata': Error reading array of size 1768180577, only > 167 bytes available > > I wonder if there is any work around? > > Thanks, > > J >> >>
Re: can kafka 10 stream API read the topic from a Kafka 9 cluster?
What I understand is the Kafka has following apis 1. Core broker APIs which runs the server. (Heart of kafka) 2. Producer API 3. Consumer API 4. Connect API 5. Streams API All four use broker for different purpose. There should be some sort of compatibility matrix. Kafka server is something central to all applications and one where frequent upgrades are not possible in a production environment. So every release of other future APIs should have some backward compatibility with Kafka server. Like x.y.z.w server should atleast be compatible with x.y.**.** APIs. So any new release x.y.**.** should make sure that they are compatible oldest x.y.**.** release atleast. And perhaps some sort of compatibility matrix page should be made available. Just my 2 cents. Thanks Sachin On Sat, Dec 24, 2016 at 12:05 AM, Hans Jespersen wrote: > No. All Java clients (including Streams) need to be the same version (or > lower) as the brokers they connect to. > > -hans > > > > > > On Dec 23, 2016, at 1:03 AM, Sachin Mittal wrote: > > > > Is Kafka streams 0.10.2.0-SNAPSHOT compatible with 0.10.0.1 broker. > > I was facing broker connect issue and it just did not work. > > > > Thanks > > Sachin > > > > > > On Fri, Dec 23, 2016 at 4:25 AM, wrote: > > > >> Kafka clients (currently) do not work against older Kafka > brokers/servers > >> so you have no other option but to upgrade to a 0.10.1.0 or higher Kafka > >> broker. > >> > >> -hans > >> > >>> On Dec 22, 2016, at 2:25 PM, Joanne Contact > >> wrote: > >>> > >>> Hello I have a program which requires 0.10.1.0 streams API. The jar is > >>> packaged by maven with all dependencies. I tried to consume a Kafka > >>> topic spit from a Kafka 9 cluster. > >>> > >>> It has such error: > >>> org.apache.kafka.common.protocol.types.SchemaException: Error reading > >>> field 'topic_metadata': Error reading array of size 1768180577, only > >>> 167 bytes available > >>> > >>> I wonder if there is any work around? > >>> > >>> Thanks, > >>> > >>> J > >> > >
Re: can kafka 10 stream API read the topic from a Kafka 9 cluster?
No. All Java clients (including Streams) need to be the same version (or lower) as the brokers they connect to. -hans > On Dec 23, 2016, at 1:03 AM, Sachin Mittal wrote: > > Is Kafka streams 0.10.2.0-SNAPSHOT compatible with 0.10.0.1 broker. > I was facing broker connect issue and it just did not work. > > Thanks > Sachin > > > On Fri, Dec 23, 2016 at 4:25 AM, wrote: > >> Kafka clients (currently) do not work against older Kafka brokers/servers >> so you have no other option but to upgrade to a 0.10.1.0 or higher Kafka >> broker. >> >> -hans >> >>> On Dec 22, 2016, at 2:25 PM, Joanne Contact >> wrote: >>> >>> Hello I have a program which requires 0.10.1.0 streams API. The jar is >>> packaged by maven with all dependencies. I tried to consume a Kafka >>> topic spit from a Kafka 9 cluster. >>> >>> It has such error: >>> org.apache.kafka.common.protocol.types.SchemaException: Error reading >>> field 'topic_metadata': Error reading array of size 1768180577, only >>> 167 bytes available >>> >>> I wonder if there is any work around? >>> >>> Thanks, >>> >>> J >>
Re: can kafka 10 stream API read the topic from a Kafka 9 cluster?
Is Kafka streams 0.10.2.0-SNAPSHOT compatible with 0.10.0.1 broker. I was facing broker connect issue and it just did not work. Thanks Sachin On Fri, Dec 23, 2016 at 4:25 AM, wrote: > Kafka clients (currently) do not work against older Kafka brokers/servers > so you have no other option but to upgrade to a 0.10.1.0 or higher Kafka > broker. > > -hans > > > On Dec 22, 2016, at 2:25 PM, Joanne Contact > wrote: > > > > Hello I have a program which requires 0.10.1.0 streams API. The jar is > > packaged by maven with all dependencies. I tried to consume a Kafka > > topic spit from a Kafka 9 cluster. > > > > It has such error: > > org.apache.kafka.common.protocol.types.SchemaException: Error reading > > field 'topic_metadata': Error reading array of size 1768180577, only > > 167 bytes available > > > > I wonder if there is any work around? > > > > Thanks, > > > > J >
Re: can kafka 10 stream API read the topic from a Kafka 9 cluster?
Kafka clients (currently) do not work against older Kafka brokers/servers so you have no other option but to upgrade to a 0.10.1.0 or higher Kafka broker. -hans > On Dec 22, 2016, at 2:25 PM, Joanne Contact wrote: > > Hello I have a program which requires 0.10.1.0 streams API. The jar is > packaged by maven with all dependencies. I tried to consume a Kafka > topic spit from a Kafka 9 cluster. > > It has such error: > org.apache.kafka.common.protocol.types.SchemaException: Error reading > field 'topic_metadata': Error reading array of size 1768180577, only > 167 bytes available > > I wonder if there is any work around? > > Thanks, > > J
can kafka 10 stream API read the topic from a Kafka 9 cluster?
Hello I have a program which requires 0.10.1.0 streams API. The jar is packaged by maven with all dependencies. I tried to consume a Kafka topic spit from a Kafka 9 cluster. It has such error: org.apache.kafka.common.protocol.types.SchemaException: Error reading field 'topic_metadata': Error reading array of size 1768180577, only 167 bytes available I wonder if there is any work around? Thanks, J