[
https://issues.apache.org/jira/browse/KAFKA-6453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16959244#comment-16959244
]
Boyang Chen commented on KAFKA-6453:
------------------------------------
Need to have both state store section and DSL section to reflect changes, such
as the reasoning of adding timestamp to state store for better:
# join semantic, previously we don't have timestamp in the state store, so
out-of-order join could not be caught. Now we could take max
# aggregation semantic, the out of order agg is also possible to push the
result backward. With timestamp, it's more defensive.
Window store and timestamped kv store are also recommended to be put in the
state store section.
> Reconsider timestamp propagation semantics
> ------------------------------------------
>
> Key: KAFKA-6453
> URL: https://issues.apache.org/jira/browse/KAFKA-6453
> Project: Kafka
> Issue Type: Improvement
> Components: streams
> Reporter: Matthias J. Sax
> Assignee: Victoria Bialas
> Priority: Major
> Labels: needs-kip
>
> Atm, Kafka Streams only has a defined "contract" about timestamp propagation
> at the Processor API level: all processor within a sub-topology, see the
> timestamp from the input topic record and this timestamp will be used for all
> result record when writing them to an topic, too.
> The DSL, inherits this "contract" atm.
> From a DSL point of view, it would be desirable to provide a different
> contract to the user. To allow this, we need to do the following:
> - extend Processor API to allow manipulation timestamps (ie, a Processor can
> set a new timestamp for downstream records)
> - define a DSL "contract" for timestamp propagation for each DSL operator
> - document the DSL "contract"
> - implement the DSL "contract" using the new/extended Processor API
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
