Appropriately timed question! Here is the PR that adds a real unit test for Kafka stream in Spark Streaming. Maybe this will help!
https://github.com/apache/spark/pull/1751/files On Mon, Aug 4, 2014 at 6:30 PM, JiajiaJing <[email protected]> wrote: > Hello Spark Users, > > I have a spark streaming program that stream data from kafka topics and > output as parquet file on HDFS. > Now I want to write a unit test for this program to make sure the output > data is correct (i.e not missing any data from kafka). > However, I have no idea about how to do this, especially how to mock a kafka > topic. > Can someone help me with this? > > Thank you very much! > > Best Regards, > > Jiajia > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Unit-Test-for-Spark-Streaming-tp11394.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
