Hi all,

We are getting collisions with subject names in our schema registry due to 
state stores that are holding Avro events:

"KSTREAM-JOINOTHER-0000000007-store-value",
  "KSTREAM-JOINOTHER-0000000006-store-value",
  "KSTREAM-JOINOTHER-0000000005-store-value",
  "KSTREAM-OUTEROTHER-0000000005-store-value",
  "KSTREAM-OUTEROTHER-0000000006-store-value",
  "KSTREAM-JOINTHIS-0000000005-store-value",
  "KSTREAM-JOINTHIS-0000000004-store-value",
  "KSTREAM-JOINTHIS-0000000006-store-value",

As you can see these are not prepended by their app id so any similarly 
constructed topology in another Streams application, that has different events 
will lead to conflicting schemas. Is this a bug, intended or user error?

I can see an advantage of the schema registry getting involved here but not if 
this happens, I see a few work arounds but unsure if any are recommended or 
there is something I have missed:

  1.  Use an ObjectSerde / Serializable (though then lose the Avro 
compatibility goodness)
  2.  Create a modified SpecificAvroSerde that includes the schema with each 
message
  3.  Anyway to influence the subject name?
  4.  Some form of local persistent schema registry for this purpose

I am not sure if anything changed with how Serdes were passed to RocksDB - I 
intend to deepdive into the source later at my next opportunity.
I did notice non avro messages in our topologies started throwing exceptions 
(not instances of SpecificRecord) since we moved to 0.10.2.0 so I suspect there 
was a plain Object serializer and now they use the topology defaults if you do 
not supply a serde.

Thanks
Adrian

Reply via email to