Github user BryanCutler commented on the issue:
https://github.com/apache/spark/pull/21546
@HyukjinKwon I redid the benchmarks for `toPandas` with the current code
and updated the description. It's not a huge speedup now, but definitely does
improve some. I'll also followup with another PR with the out-of-order batches
to improve this even further. Let me know if this looks ok to you (pending
tests). Thanks!
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
