Re: Sorting each partitions and writing to CSVs
For those interested, after digging further, I was able to consistently reproduce the issue with a synthetic dataset. My findings are documented here: https://gist.github.com/igozali/d327a85646abe7ab10c2ae479bed431f -- Regards, Ivan Gozali Lecida Email: i...@lecida.com On Wed, Jan 18, 2017
Sorting each partitions and writing to CSVs
tially a bug in Spark. I'm using Spark 2.0.2 with Python 2.7.12. Any advice would be very much appreciated! -- Regards, Ivan Gozali