Hi Adam, I haven't looked into your project yet, but just wanted to mention this to see if it explains your observations.
TopologyTestDriver processes every input record fully and synchronously. So, when you pipe an input record in, it traverses all internal processing, including intermediate topics, until all results are written to sink topics (or the end of the topology is reached). This may indeed result in a different order of output records than you'd observe with Kafka Streams, which would instead write to the intermediate topic in one thread and then process it in a separate thread. Does that add up with what you're seeing? Thanks, -John On Wed, Sep 11, 2019 at 6:40 AM Adam Domanski <adoman...@gmail.com> wrote: > > Hi, > > I probably found a bug in TopologyTestDriver for quite non-trivial Kafka > Streams topology. > > The streaming logic is the following: There is a concept of children and > parents. Children are aggregated under parent. Some children of master > parent can send poison pills to other parents. Such parents die but already > aggregated kids should go then to the master. Application remembers killed > parents in a local store. If a new kid comes, the store is checked and then > kid is rerouted to the master. > > I'm using intermediate topics in my topology as well as sometimes > tombstones. > > Here is Gradle (Java 11 + Kotlin) based project which demonstrate the > issue: https://github.com/czterocyty/kafka_streams_test_case > > You can see that few tests fail there as I expect that latest output Parent > records have the latest generation. > > So seems that TopologyTestDriver does not care about the order of output > records. > > Best regards, > Adam Domański
