We are seeing odd behavior that we need to understand. We are getting
IncompatibleClassChangeErrors and I know that's related to a Scala version
mismatch. What's not clear, however, is where or why the mismatch is occurring.
We know up front that there were occasions where we ran apps that had Scala
2.10 dependencies with the 0.9 consumer. We also plugged in a metrics reporter
at the broker with 2.10 scala dependencies. Both of these situations produced
the class change errors and they both appear to be around the consumer offsets
topic. Is there something under the hood that is serializing a scala object
that would cause this issue? The only fix appears to be blowing away all data
in the __consumer_offsets topic and starting over with 2.11 clients. Is this
expected behavior? Seems like a weakness if so because we have no control, in
some cases, what version of Scala a client might use.