There is also the Spark Testing Base package which is on spark-packages.org and hides the ugly bits (it's based on the existing streaming test code but I cleaned it up a bit to try and limit the number of internals it was touching).
On Sunday, March 1, 2015, Marcin Kuthan <marcin.kut...@gmail.com> wrote: > I have started using Spark and Spark Streaming and I'm wondering how do > you test your applications? Especially Spark Streaming application with > window based transformations. > > After some digging I found ManualClock class to take full control over > stream processing. Unfortunately the class is not available outside > spark.streaming package. Are you going to expose the class for other > developers as well? Now I have to use my custom wrapper under > spark.streaming package. > > My Spark and Spark Streaming unit tests strategies are documented here: > http://mkuthan.github.io/blog/2015/03/01/spark-unit-testing/ > > Your feedback is more than appreciated. > > Marcin > > -- Cell : 425-233-8271
