Something more on the lines of integration I believe. Run one or more Spark jobs and verify the output results. If this makes sense.
I am very new to the world of Spark. We want to include pipeline testing from the get go. I will check out spark-testing-base. Thanks. From: Holden Karau [mailto:hol...@pigscanfly.ca] Sent: Monday, May 21, 2018 11:32 AM To: Steve Pruitt <bpru...@opentext.com> Cc: user@spark.apache.org Subject: [EXTERNAL] - Re: testing frameworks So I’m biased as the author of spark-testing-base but I think it’s pretty ok. Are you looking for unit or integration or something else? On Mon, May 21, 2018 at 5:24 AM Steve Pruitt <bpru...@opentext.com<mailto:bpru...@opentext.com>> wrote: Hi, Can anyone recommend testing frameworks suitable for Spark jobs. Something that can be integrated into a CI tool would be great. Thanks. -- Twitter: https://twitter.com/holdenkarau<https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_holdenkarau&d=DwMFaQ&c=ZgVRmm3mf2P1-XDAyDsu4A&r=ksx9qnQFG3QvxkP54EBPEzv1HHDjlk-MFO-7EONGCtY&m=YTdxEm6qmXE1TQvlRzPccMkNLcynfxhC32Uj91HcaXA&s=a_ORg1aB6eKT2ZYxtSJw3oOQnHmi07gjf9whuROeNYw&e=>
