Hi! Let me draw your attention to Holden's* spark-testing-base* project. The documentation is at https://github.com/holdenk/spark-testing-base/wiki.
As I usually write test for spark internal features I haven't needed to test so high level. But I am interested about your experiences. Best regards, Attila On Mon, Mar 22, 2021 at 4:34 PM Nicholas Gustafson <njgustaf...@gmail.com> wrote: > I've found pytest works well if you're using PySpark. Though if you have a > lot of tests, running them all can be pretty slow. > > On Mon, Mar 22, 2021 at 6:32 AM Amit Sharma <resolve...@gmail.com> wrote: > >> Hi, can we write unit tests for spark code. Is there any specific >> framework? >> >> >> Thanks >> Amit >> >
