Milan Baran created HIVE-21218:
----------------------------------
Summary: KafkaSerDe doesn't support topics created via Confluent
Avro serializer
Key: HIVE-21218
URL: https://issues.apache.org/jira/browse/HIVE-21218
Project: Hive
Issue Type: Bug
Components: kafka integration, Serializers/Deserializers
Affects Versions: 3.1.1
Reporter: Milan Baran
According to [Google
groups|https://groups.google.com/forum/#!topic/confluent-platform/JYhlXN0u9_A]
the Confluent avro serialzier uses propertiary format for kafka value -
<magic_byte 0x00><4 bytes of schema ID><regular avro bytes for object that
conforms to schema>.
This format does not cause any problem for Confluent kafka deserializer which
respect the format however for hive kafka handler its bit a problem to
correctly deserialize kafka value, because Hive uses custom deserializer from
bytes to objects and ignores kafka consumer ser/deser classes provided via
table property.
It would be nice to support Confluent format with magic byte.
Also it would be great to support Schema registry as well.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
