[
https://issues.apache.org/jira/browse/FLINK-4050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15330889#comment-15330889
]
Elias Levy commented on FLINK-4050:
-----------------------------------
I can get around it, it's just not very pretty code.
A no-op KeyedSerializationSchema / SerializationSchema would work. Not sure if
that would mean changing the return type of serializeValue / serializeKey to
Object, adding a new field to the interface to point out it is a no-op, or
allowing the passing in of a null for the serialization schema parameter to
FlinkKafkaProducer.
That said, there would still be a need to add a new method to the interface to
separate serialization from key extraction, which right now
KeyedSerializationSchema.serializeKey is expected to do. Although, doing so
would be backwards compatible.
> FlinkKafkaProducer API Refactor
> -------------------------------
>
> Key: FLINK-4050
> URL: https://issues.apache.org/jira/browse/FLINK-4050
> Project: Flink
> Issue Type: Improvement
> Components: Kafka Connector
> Affects Versions: 1.0.3
> Reporter: Elias Levy
>
> The FlinkKafkaProducer API seems more difficult to use than it should be.
> The API requires you pass it a SerializationSchema or a
> KeyedSerializationSchema, but the Kafka producer already has a serialization
> API. Requiring a serializer in the Flink API precludes the use of the Kafka
> serializers. For instance, they preclude the use of the Confluent
> KafkaAvroSerializer class that makes use of the Confluent Schema Registry.
> Ideally, the serializer would be optional, so as to allow the Kafka producer
> serializers to handle the task.
> In addition, the KeyedSerializationSchema conflates message key extraction
> with key serialization. If the serializer were optional, to allow the Kafka
> producer serializers to take over, you'd still need to extract a key from the
> message.
> And given that the key may not be part of the message you want to write to
> Kafka, an upstream step may have to package the key with the message to make
> both available to the sink, for instance in a tuple. That means you also need
> to define a method to extract the message to write to Kafka from the element
> passed into the sink by Flink.
> In summary, there should be separation of extraction of the key and message
> from the element passed into the sink from serialization, and the
> serialization step should be optional.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
