Re: Mocking pyspark read writes

2020-07-07 Thread Jörn Franke 
Write to a local temp directory via file:// ?

> Am 07.07.2020 um 20:07 schrieb Dark Crusader :
> 
> 
> Hi everyone,
> 
> I have a function which reads and writes a parquet file from HDFS. When I'm 
> writing a unit test for this function, I want to mock this read & write.
> 
> How do you achieve this? 
> Any help would be appreciated. Thank you.
> 

-
To unsubscribe e-mail: user-unsubscr...@spark.apache.org

Mocking pyspark read writes

2020-07-07 Thread Dark Crusader 
Hi everyone,

I have a function which reads and writes a parquet file from HDFS. When I'm
writing a unit test for this function, I want to mock this read & write.

How do you achieve this?
Any help would be appreciated. Thank you.