Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21546
It's not a bug. Here's what's going on now:
He proposed a way to write out regardless of partition order, I suggested
to revert that part back (see
https://github.com/apache/spark/pull/21546#discussion_r204324646) since the
code path looks being complicated and looks that can orthogonally proceed,
which can be also applied to other PySpark code path.
This one still has some improvements (see
https://github.com/apache/spark/pull/21546#issuecomment-416373315).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
