[ https://issues.apache.org/jira/browse/KAFKA-7653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Matthias J. Sax updated KAFKA-7653: ----------------------------------- Description: Implicit resolution/conversion of Serdes/Consumed etc is a big improvement for the Scala Streams API. However in cases where a user needs to differentiate between Key and Value serializer functionality (i.e. using the Schema Registry), implicit resolution doesn't help and could cause issues. e.g. {code:java} case class MouseClickEvent(pageId: Long, userId: String) builder // Long serde taken from implicit scope configured with // `isKey` = true .stream[Long, MouseClickEvent]("mouse-clicks") .selectKey((_,v) => v.userId) .groupByKey .aggregate(() => 0L, (_: String, mce: MouseClickEvent, count: Long) => count + 1) .toStream // Same Long serde taken from implicit scope configured with // `isKey` = true, even thought the `Long` value in this case // will be the Value .to("mouse-clicks-by-user") {code} It would be ideal if Key and Value Serde/SerdeWrapper types/type classes could be introduced to overcome this limitation. KIP-513: [https://cwiki.apache.org/confluence/display/KAFKA/KIP-513%3A+Distinguish+between+Key+and+Value+serdes+in+scala+wrapper+library+for+kafka+streams] was: Implicit resolution/conversion of Serdes/Consumed etc is a big improvement for the Scala Streams API. However in cases where a user needs to differentiate between Key and Value serializer functionality (i.e. using the Schema Registry), implicit resolution doesn't help and could cause issues. e.g. {code:java} case class MouseClickEvent(pageId: Long, userId: String) builder // Long serde taken from implicit scope configured with // `isKey` = true .stream[Long, MouseClickEvent]("mouse-clicks") .selectKey((_,v) => v.userId) .groupByKey .aggregate(() => 0L, (_: String, mce: MouseClickEvent, count: Long) => count + 1) .toStream // Same Long serde taken from implicit scope configured with // `isKey` = true, even thought the `Long` value in this case // will be the Value .to("mouse-clicks-by-user") {code} It would be ideal if Key and Value Serde/SerdeWrapper types/type classes could be introduced to overcome this limitation. > Streams-Scala: Add type level differentiation for Key and Value serdes. > ----------------------------------------------------------------------- > > Key: KAFKA-7653 > URL: https://issues.apache.org/jira/browse/KAFKA-7653 > Project: Kafka > Issue Type: Improvement > Components: streams > Reporter: Mark Tranter > Assignee: Mark Tranter > Priority: Minor > Labels: kip, scala > > Implicit resolution/conversion of Serdes/Consumed etc is a big improvement > for the Scala Streams API. However in cases where a user needs to > differentiate between Key and Value serializer functionality (i.e. using the > Schema Registry), implicit resolution doesn't help and could cause issues. > e.g. > {code:java} > case class MouseClickEvent(pageId: Long, userId: String) > builder > // Long serde taken from implicit scope configured with > // `isKey` = true > .stream[Long, MouseClickEvent]("mouse-clicks") > .selectKey((_,v) => v.userId) > .groupByKey > .aggregate(() => 0L, (_: String, mce: MouseClickEvent, count: Long) => > count + 1) > .toStream > // Same Long serde taken from implicit scope configured with > // `isKey` = true, even thought the `Long` value in this case > // will be the Value > .to("mouse-clicks-by-user") > {code} > It would be ideal if Key and Value Serde/SerdeWrapper types/type classes > could be introduced to overcome this limitation. > KIP-513: > [https://cwiki.apache.org/confluence/display/KAFKA/KIP-513%3A+Distinguish+between+Key+and+Value+serdes+in+scala+wrapper+library+for+kafka+streams] -- This message was sent by Atlassian Jira (v8.20.10#820010)