And now (before 1am California time :p) there is a new version of spark-testing base which adds a java base class for streaming tests. I noticed you were using 1.3 so I put in the effort to make this release for Spark 1.3 to 1.5 inclusive).
On Wed, Oct 21, 2015 at 4:16 PM, Holden Karau <hol...@pigscanfly.ca> wrote: > > > On Wednesday, October 21, 2015, Mark Vervuurt <m.a.vervu...@gmail.com> > wrote: > >> Hi Holden, >> >> Thanks for the information, I think that a Java Base Class in order to >> test SparkStreaming using Java would be useful for the community. >> Unfortunately not all of our customers are willing to use Scala or Python. >> > Sounds reasonable, I'll add it this week. > >> >> If i am not wrong it’s 4:00 AM for you in California ;) >> >> Yup, I'm not great a regular schedules but I make up for it by doing > stuff when I've had too much coffee to sleep :p > >> Regards, >> Mark >> >> On 21 Oct 2015, at 12:42, Holden Karau <hol...@pigscanfly.ca> wrote: >> >> >> >> On Wednesday, October 21, 2015, Mark Vervuurt <m.a.vervu...@gmail.com> >> wrote: >> >>> Hi Everyone, >>> >>> I am busy trying out ‘Spark-Testing-Base >>> <https://github.com/holdenk/spark-testing-base>’. I have the following >>> questions? >>> >>> >>> - Can you test Spark Streaming Jobs using Java? >>> >>> The current base class for testing streaming jobs is implemented using a >> Scala test library (and one in Python too), I can add one using a junit >> base for streaming if it would be useful for you. >> >>> >>> - Can I use Spark-Testing-Base 1.3.0_0.1.1 together with Spark 1.3.1? >>> >>> You should be able to, the API changes were small enough I didn't >> publish a seperate package, but if you run into any issues let me know. >> >>> >>> >>> >>> Thanks. >>> >>> Greetings, >>> Mark >>> >> >> >> -- >> Cell : 425-233-8271 >> Twitter: https://twitter.com/holdenkarau >> >> >> -- Cell : 425-233-8271 Twitter: https://twitter.com/holdenkarau
