We've built a driver program which kinda falls along approach (1) listed in your email.
The driver program accepts a custom task object and has a way to inject data -> which in turn invokes the process method. For now we're assuming logical time and use the frequency of process() invocations to deduce when to invoke the window() method (for eg: invoke window once for every 4 calls to process). We've also built our own Incoming and Outgoing envelope -> which is just in the form of a Java List. This is how the result is evaluated (either get the full result list or define a callback which is invoked every time a collector.send is called). Its still work in progress and the goal is to make unit testing along the lines of Storm unit tests. On Tue, Jun 16, 2015 at 5:17 PM, Chris Riccomini <criccom...@apache.org> wrote: > Hey Tim, > > This is a really good discussion to have. The testing that I've seen with > Samza falls into two categories: > > 1. Instantiate your StreamTask, and mock all params in the process()/init() > methods. > 2. A mini-ontegration test that starts ZooKeeper, and Kafka, and feeds > messages into a topic, and validates it gets messages back out from the > output topic. > 3. A full blown integration test that uses Zopkio. > > For an example of (2), in practice, have a look at TestStatefuleTask: > > > > https://git-wip-us.apache.org/repos/asf?p=samza.git;a=blob;f=samza-test/src/test/scala/org/apache/samza/test/integration/TestStatefulTask.scala;h=ea702a919348305ff95ce0b4ca1996a13aff04ec;hb=HEAD > > As you can see, writing this kind of integration test can be a bit painful. > > (3) is documented here: > > http://samza.apache.org/contribute/tests.html > > Another way to test would be to start a full-blown container using > ThreadJobFactory/ProcessJobFactory, but use a MockSystemFactory to mock out > the system consumer/system producer. > > Has anyone else tested Samza in other ways? > > Cheers, > Chris > > On Tue, Jun 16, 2015 at 11:00 AM, Tim Williams <william...@gmail.com> > wrote: > > > I'm learning samza by the hello-samza project and notice the lack of > > tests. Where's a good place to learn how folks are properly testing > > things written with samza? > > > > Thanks, > > --tim > > > -- Thanks and regards Chinmay Soman
