[
https://issues.apache.org/jira/browse/NIFI-2515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15412225#comment-15412225
]
Joseph Witt commented on NIFI-2515:
-----------------------------------
Had a chance to speak with Oleg. The discussion centered around how both Kafka
and NiFi treat data/payload/messages/whatever as bag of bits and that indeed it
is the clients which handle actually understanding those bits. The kafka
producer/consumer API allows you to register key and value serializers and
deserializers so you can interact with the client API using the desired object
type and Kafka will ensure the proper ser/de calls are made to convert bytes to
objects and objects to bytes. In the case of NiFi this
serialization/deserialization process has already occurred by the time we're
interacting with Kafka so we don't need additional mechanisms to accomplish
this.
The solution here then is to ensure that we're always using the default byte[]
serde and if the user attempts to configure the processor to use an alternative
serde we will flag the processor as invalid.
If Kafka does end up saving/sharing this information between producers and
consumers then we'll want to support/bridge that but probably the schema
registry model is a better answer here anyway.
Long story short: Easy thing to correct to regain correct behavior and the
correct step is to ensure we're using raw byte[] ser/de and not something else.
> Setting key.serializer in publish kafka leads to
> org.apache.kafka.common.errors.SerializationException
> ------------------------------------------------------------------------------------------------------
>
> Key: NIFI-2515
> URL: https://issues.apache.org/jira/browse/NIFI-2515
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
> Affects Versions: 1.0.0-Beta
> Reporter: Arpit Gupta
> Assignee: Oleg Zhurakousky
> Priority: Minor
> Fix For: 1.0.0, 0.8.0
>
>
> If one sets key.serializer in the publish kafka processor we see error
> {code}
> 4895-9b14-9e8b1d88755e}, body={topics=[putMessages-83cf2948-b110-2016-08-07
> 17:11:56,191 ERROR [Timer-Driven Process Thread-7]
> o.a.n.p.kafka.pubsub.PublishKafka
> PublishKafka[id=65fd2baf-0156-1000-ffff-fffff6bf89be]
> PublishKafka[id=65fd2baf-0156-1000-ffff-fffff6bf89be] failed to process due
> to org.apache.kafka.common.errors.SerializationException: Can't convert key
> of class [B to class org.apache.kafka.common.serialization.IntegerSerializer
> specified in key.serializer; rolling back session:
> org.apache.kafka.common.errors.SerializationException: Can't convert key of
> class [B to class org.apache.kafka.common.serialization.IntegerSerializer
> specified in key.serializer
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
