Re: Adding Spark Testing functionality

2015-10-12 Thread Holden Karau
So here is a quick description of the current testing bits (I can expand on it if people are interested) http://bit.ly/pandaPandaPanda . On Tue, Oct 6, 2015 at 3:49 PM, Holden Karau wrote: > I'll put together a google doc and send that out (in the meantime a quick > guide

Adding Spark Testing functionality

2015-10-06 Thread Holden Karau
Hi Spark Devs, So this has been brought up a few times before, and generally on the user list people get directed to use spark-testing-base. I'd like to start moving some of spark-testing-base's functionality into Spark so that people don't need a library to do what is (hopefully :p) a very

Re: Adding Spark Testing functionality

2015-10-06 Thread Patrick Wendell
Hey Holden, It would be helpful if you could outline the set of features you'd imagine being part of Spark in a short doc. I didn't see a README on the existing repo, so it's hard to know exactly what is being proposed. As a general point of process, we've typically avoided merging modules into

Re: Adding Spark Testing functionality

2015-10-06 Thread Holden Karau
I'll put together a google doc and send that out (in the meantime a quick guide of sort of how the current package can be used is in the blog post I did at http://blog.cloudera.com/blog/2015/09/making-apache-spark-testing-easy-with-spark-testing-base/ ) If people think its better to keep as a