[
https://issues.apache.org/jira/browse/KAFKA-3543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15236434#comment-15236434
]
Guozhang Wang commented on KAFKA-3543:
--------------------------------------
Yeah, we should definitely improve our documentation regarding serdes. In
general the documentation website is not very well formatted: it is just a
single VERY big page, and hence very hard for people to search and browse. We
are thinking about reformatting the Apache doc website and add more contents
about different modules like Connect and Streams.
I think it is actually doable to add a "flatTransform" function along with
"transform" in KStreams. I will update this ticket to reflect this feature
request.
> Allow a variant of transform() which can emit multiple values
> -------------------------------------------------------------
>
> Key: KAFKA-3543
> URL: https://issues.apache.org/jira/browse/KAFKA-3543
> Project: Kafka
> Issue Type: Improvement
> Components: streams
> Affects Versions: 0.10.0.0
> Reporter: Greg Fodor
> Assignee: Guozhang Wang
>
> Right now it seems that if you want to apply an arbitrary stateful
> transformation to a stream, you either have to use a TransformerSupplier or
> ProcessorSupplier sent to transform() or process(). The custom processor will
> allow you to emit multiple new values, but the process() method currently
> terminates that branch of the topology so you can't apply additional data
> flow. transform() lets you continue the data flow, but forces you to emit a
> single value for every input value.
> (It actually doesn't quite force you to do this, since you can hold onto the
> ProcessorContext and emit multiple, but that's probably not the ideal way to
> do it :))
> It seems desirable to somehow allow a transformation that emits multiple
> values per input value. I'm not sure of the best way to factor this inside of
> the current TransformerSupplier/Transformer architecture in a way that is
> clean and efficient -- currently I'm doing the workaround above of just
> calling forward() myself on the context and actually emitting dummy values
> which are filtered out downstream.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
