Re: Time of derived records in Kafka Streams

2016-09-19 Thread Eno Thereska
Hi Elias, - out of order records: the timestamp is that of the out of order record, i.e., time goes backwards sometimes - joins: the same, the timestamp could be that of either record. We'll update the docs, thanks for your question. Eno > On 17 Sep 2016, at 00:43, Elias Levy

Re: Time of derived records in Kafka Streams

2016-09-16 Thread Elias Levy
On Sat, Sep 10, 2016 at 9:17 AM, Eno Thereska wrote: > > For aggregations, the timestamp will be that of the latest record being > aggregated. > How does that account for out of order records? What about kstream-kstream joins? The output from the join could be

Re: Time of derived records in Kafka Streams

2016-09-10 Thread Eno Thereska
Hi Elias, Good question. The general answer is that each time a record is output, the timestamp is that of the current Kafka Streams task that processes it, so it's the internal Kafka Streams time. If the Kafka Streams task is processing records with event time, the timestamp at any point is

Time of derived records in Kafka Streams

2016-09-09 Thread Elias Levy
The Kafka Streams documentation discussed how to assign timestamps to records received from source topic via TimestampExtractor. But neither the Kafka nor the Confluent documentation on Kafka Streams explain what timestamp is associated with a record that has been transformed. What timestamp is