I have started using Spark and Spark Streaming and I'm wondering how do you test your applications? Especially Spark Streaming application with window based transformations.
After some digging I found ManualClock class to take full control over stream processing. Unfortunately the class is not available outside spark.streaming package. Are you going to expose the class for other developers as well? Now I have to use my custom wrapper under spark.streaming package. My Spark and Spark Streaming unit tests strategies are documented here: http://mkuthan.github.io/blog/2015/03/01/spark-unit-testing/ Your feedback is more than appreciated. Marcin
