Re: Exception while deserializing in kafka streams

2016-09-27 Thread Walter rakoff
Ah, that was it. I was passing the same Serde while creating the topology. It works after I removed it. Thanks! Walter On Mon, Sep 26, 2016 at 1:16 PM, Guozhang Wang wrote: > Hi Walter, > > One thing I can think of is that, if you pass the serde object as part of > your topology definition, in

Re: Exception while deserializing in kafka streams

2016-09-26 Thread Guozhang Wang
Hi Walter, One thing I can think of is that, if you pass the serde object as part of your topology definition, instead of passing the serde class in the config, then these serde objects will not be auto configured and hence for your case the schema registry client will not be constructed and initi

Re: Exception while deserializing in kafka streams

2016-09-22 Thread Guozhang Wang
Hi Walter, I downloaded the 0.10.0 jar and verified that the configure() function is auto-triggered when you get the serde classes from `context.keySerde / valueSerde`, which is auto-triggered if you use the DSL. And your Scala code is the same as to our examples code: https://github.com/confluen

Re: Exception while deserializing in kafka streams

2016-09-22 Thread Walter rakoff
Guozhang, I tried your suggestion. Below is the log from Serde, Serializer & Deserializer. Confirmed that KafkaAvroDeserializer.configure does get invoked. Line 379: 16/09/22 15:28:46 WARN GenericAvroSerdeWithSchemaRegistry: In > configure {num.standby.replicas=1, replication.factor=3, > commit.i

Re: Exception while deserializing in kafka streams

2016-09-19 Thread Guozhang Wang
Hello Walter, The WARN log entry should not be the cause of this issue. I double checked the 0.10.0.0 release and this issue should not really happen, so your observation is a bit weird to me. Could your add a log entry in the `configure` function which constructs the registry client to make sure

Re: Exception while deserializing in kafka streams

2016-09-16 Thread Walter rakoff
Guozhang, Any clues on this one? Walter On Wed, Sep 14, 2016 at 9:46 PM, Walter rakoff wrote: > Guozhang, > > I am using 0.10.0.0. Could the below log be the cause? > > 16/09/14 17:24:35 WARN ConsumerConfig: The configuration > schema.registry.url = http://192.168.50.6:

Re: Exception while deserializing in kafka streams

2016-09-14 Thread Walter rakoff
Guozhang, I am using 0.10.0.0. Could the below log be the cause? 16/09/14 17:24:35 WARN ConsumerConfig: The configuration schema.registry.url = http://192.168.50.6: 8081 was supplied but isn't a known config. 16/09/14 17:24:35 INFO AppInfoParser: Kafka version : 0.10.0.

Re: Exception while deserializing in kafka streams

2016-09-14 Thread Guozhang Wang
Hello Walter, Which version of Kafka were you using? I ask this because there was a bug causing the serde passed through config to NOT being configured when constructed: https://issues.apache.org/jira/browse/KAFKA-3639 Which is fixed in the 0.10.0.0 release, which means you will only hit it if