[
https://issues.apache.org/jira/browse/KAFKA-7653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16699614#comment-16699614
]
Mark Tranter commented on KAFKA-7653:
-------------------------------------
Thanks [~vvcephei],
Different serde instances for keys/values is exactly what we want. The problem
is that with the current Scala API, there is no way for the compiler to
garauntee different implicit instances for the same K/V types.
i.e.
{code}
class KStreamBuilder {
def stream[K,V](topic: String)(implicit keySerde: Serde[K], valueSerde:
Serde[V]) ...
}{code}
If `K` and `V` are the same type, the compiler will use the same implicit
definition. We cant guarantee that this definition will return different
instances which makes auto-config problematic.
I can also imagine a scenario like this...
{code}
implicit val keySerde: Serde[String] = new SerdeImplA[String](... )
implicit val valueSerde: Serde[String] = new SerdeImplB[String](...)
// Compilation error. Ambiguous Implicit definition.
builder.stream[String, String]("my-topic")
{code}
Currently there would be no way of doing this with the existing scala API
(apart from explicit passing of serdes. Which negates the need for the Scala
API).
The only way around this - that I can see - is to provide K/V serde
differentiation at the type level.
> Streams-Scala: Add type level differentiation for Key and Value serdes.
> -----------------------------------------------------------------------
>
> Key: KAFKA-7653
> URL: https://issues.apache.org/jira/browse/KAFKA-7653
> Project: Kafka
> Issue Type: Improvement
> Components: streams
> Reporter: Mark Tranter
> Assignee: Mark Tranter
> Priority: Minor
> Labels: scala
>
> Implicit resolution/conversion of Serdes/Consumed etc is a big improvement
> for the Scala Streams API. However in cases where a user needs to
> differentiate between Key and Value serializer functionality (i.e. using the
> Schema Registry), implicit resolution doesn't help and could cause issues.
> e.g.
> {code:java}
> case class MouseClickEvent(pageId: Long, userId: String)
> builder
> // Long serde taken from implicit scope configured with
> // `isKey` = true
> .stream[Long, MouseClickEvent]("mouse-clicks")
> .selectKey((_,v) => v.userId)
> .groupByKey
> .aggregate(() => 0L, (_: String, mce: MouseClickEvent, count: Long) =>
> count + 1)
> .toStream
> // Same Long serde taken from implicit scope configured with
> // `isKey` = true, even thought the `Long` value in this case
> // will be the Value
> .to("mouse-clicks-by-user")
> {code}
> It would be ideal if Key and Value Serde/SerdeWrapper types/type classes
> could be introduced to overcome this limitation.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
