Hi Jay, Thanks for the reply.
Unfortunately in our case due to legacy reasons we are using WallclockTimestampExtractor in the application for all the streams and the existing messages in the stream probably won¹t have timestamps as they are being produced by legacy clients. So the events are being ingested with processing times and it may not be able to synchronize based on the message timestamps. What do you recommend for this scenario? Rohit On 6/28/16, 5:18 PM, "Jay Kreps" <j...@confluent.io> wrote: >I think you may get this for free as Kafka Streams attempts to align >consumption across different topics/partitions by the timestamp in the >messages. So in a case where you are starting a job fresh and it has a >database changelog to consume and a event stream to consume, it will >attempt to keep the Ktable at the "time" the event stream is at. This is >only a heuristic, of course, since messages are necessarily strongly >ordered by time. I think this is likely mostly the same but slightly >better >than the bootstrap usage in Samza but also covers other cases of >alignment. > >If you want more control you can override the timestamp extractor that >associates time and hence priority for the streams: >https://kafka.apache.org/0100/javadoc/org/apache/kafka/streams/processor/T >imestampExtractor.html > >-Jay > >On Tue, Jun 28, 2016 at 2:49 PM, Rohit Valsakumar <rvalsaku...@tivo.com> >wrote: > >> Hi all, >> >> Is there a way to consume all the contents of a kafka topic into a >>KTable >> before doing a left join with another Kstream? >> >> I am looking at something that simulates a bootstrap topic in a Samza >>job. >> >> Thanks, >> Rohit Valsakumar >> >> ________________________________ >> >> This email and any attachments may contain confidential and privileged >> material for the sole use of the intended recipient. Any review, >>copying, >> or distribution of this email (or any attachments) by others is >>prohibited. >> If you are not the intended recipient, please contact the sender >> immediately and permanently delete this email and any attachments. No >> employee or agent of TiVo Inc. is authorized to conclude any binding >> agreement on behalf of TiVo Inc. by email. Binding agreements with TiVo >> Inc. may only be made by a signed written agreement. >> ________________________________ This email and any attachments may contain confidential and privileged material for the sole use of the intended recipient. Any review, copying, or distribution of this email (or any attachments) by others is prohibited. If you are not the intended recipient, please contact the sender immediately and permanently delete this email and any attachments. No employee or agent of TiVo Inc. is authorized to conclude any binding agreement on behalf of TiVo Inc. by email. Binding agreements with TiVo Inc. may only be made by a signed written agreement.
