[
https://issues.apache.org/jira/browse/BEAM-2221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eugene Kirpichov updated BEAM-2221:
-----------------------------------
Description:
readWithCoders and writeWithCoders functions are awkward because they don't
emphasize enough that coders are a poor choice for interpreting wire format.
The only reason to specify coders in KafkaIO is when coder inference from
Deserializer fails. To emphasize that, let's change the API to be
withKeyDeserializer(Deserializer) as the default choice and
withKeyDeserializerAndCoder(Deserializer,Coder) if inference fails; likewise
for value.
Remove functions using coders to interpret wire format from the API. A common
case of that is Avro and Proto - for that, introduce special helper functions,
I guess like withAvro/ProtoKey/Value(...), which under the hood may be allowed
to reuse Avro/ProtoCoder as a utility, but do not expose this fact.
was:
readWithCoders and writeWithCoders functions are awkward because they don't
emphasize enough that coders are a poor choice for interpreting wire format.
The only reason to specify coders in KafkaIO is when coder inference from
Deserializer fails. To emphasize that, let's change the API to be
withKeyDeserializer(Deserializer) as the default choice and
withKeyDeserializerAndCoder(Deserializer,Coder) if inference fails; likewise
for value.
Remove functions using coders to interpret wire format from the API. A common
case of that is Avro and Proto - for that, introduce special helper functions,
I guess like with{Avro,Proto}{Key,Value}(...), which under the hood may be
allowed to reuse Avro/ProtoCoder as a utility, but do not expose this fact.
> Make KafkaIO coder specification less awkward
> ---------------------------------------------
>
> Key: BEAM-2221
> URL: https://issues.apache.org/jira/browse/BEAM-2221
> Project: Beam
> Issue Type: Improvement
> Components: sdk-java-extensions
> Reporter: Eugene Kirpichov
> Assignee: Raghu Angadi
> Fix For: 2.0.0
>
>
> readWithCoders and writeWithCoders functions are awkward because they don't
> emphasize enough that coders are a poor choice for interpreting wire format.
> The only reason to specify coders in KafkaIO is when coder inference from
> Deserializer fails. To emphasize that, let's change the API to be
> withKeyDeserializer(Deserializer) as the default choice and
> withKeyDeserializerAndCoder(Deserializer,Coder) if inference fails; likewise
> for value.
> Remove functions using coders to interpret wire format from the API. A common
> case of that is Avro and Proto - for that, introduce special helper
> functions, I guess like withAvro/ProtoKey/Value(...), which under the hood
> may be allowed to reuse Avro/ProtoCoder as a utility, but do not expose this
> fact.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
