Hi all, whoever (Sam I think) was going to do some work on doing a template testing pipeline. I'd love to be involved, I have a current task in my day job (data engineer) to flesh out our testing how-to / best practices for Spark jobs and I think I'll be doing something very similar for the next week or 2.
I'll scrape out what i have now in the next day or so and put it up in a gist that I can share too. G On 25 April 2017 at 13:04, Holden Karau <hol...@pigscanfly.ca> wrote: > Urgh hangouts did something frustrating, updated link > https://hangouts.google.com/hangouts/_/ha6kusycp5fvzei2trhay4uhhqe > > On Mon, Apr 24, 2017 at 12:13 AM, Holden Karau <hol...@pigscanfly.ca> > wrote: > >> The (tentative) link for those interested is https://hangouts.google.com >> /hangouts/_/oyjvcnffejcjhi6qazf3lysypue . >> >> On Mon, Apr 24, 2017 at 12:02 AM, Holden Karau <hol...@pigscanfly.ca> >> wrote: >> >>> So 14 people have said they are available on Tuesday the 25th at 1PM >>> pacific so we will do this meeting then ( https://doodle.com/poll/69y6 >>> yab4pyf7u8bn ). >>> >>> Since hangouts tends to work ok on the Linux distro I'm running my >>> default is to host this as a "hangouts-on-air" unless there are alternative >>> ideas. >>> >>> I'll record the hangout and if it isn't terrible I'll post it for those >>> who weren't able to make it (and for next time I'll include more European >>> friendly time options - Doodle wouldn't let me update it once posted). >>> >>> On Fri, Apr 14, 2017 at 11:17 AM, Holden Karau <hol...@pigscanfly.ca> >>> wrote: >>> >>>> Hi Spark Users (+ Some Spark Testing Devs on BCC), >>>> >>>> Awhile back on one of the many threads about testing in Spark there was >>>> some interest in having a chat about the state of Spark testing and what >>>> people want/need. >>>> >>>> So if you are interested in joining an online (with maybe an IRL >>>> component if enough people are SF based) chat about Spark testing please >>>> fill out this doodle - https://doodle.com/poll/69y6yab4pyf7u8bn >>>> >>>> I think reasonable topics of discussion could be: >>>> >>>> 1) What is the state of the different Spark testing libraries in the >>>> different core (Scala, Python, R, Java) and extended languages (C#, >>>> Javascript, etc.)? >>>> 2) How do we make these more easily discovered by users? >>>> 3) What are people looking for in their testing libraries that we are >>>> missing? (can be functionality, documentation, etc.) >>>> 4) Are there any examples of well tested open source Spark projects and >>>> where are they? >>>> >>>> If you have other topics that's awesome. >>>> >>>> To clarify this about libraries and best practices for people testing >>>> their Spark applications, and less about testing Spark's internals >>>> (although as illustrated by some of the libraries there is some strong >>>> overlap in what is required to make that work). >>>> >>>> Cheers, >>>> >>>> Holden :) >>>> >>>> -- >>>> Cell : 425-233-8271 <(425)%20233-8271> >>>> Twitter: https://twitter.com/holdenkarau >>>> >>> >>> >>> >>> -- >>> Cell : 425-233-8271 <(425)%20233-8271> >>> Twitter: https://twitter.com/holdenkarau >>> >> >> >> >> -- >> Cell : 425-233-8271 <(425)%20233-8271> >> Twitter: https://twitter.com/holdenkarau >> > > > > -- > Cell : 425-233-8271 <(425)%20233-8271> > Twitter: https://twitter.com/holdenkarau >