Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/21546#discussion_r195824174
--- Diff: python/pyspark/sql/dataframe.py ---
@@ -2153,7 +2153,7 @@ def _collectAsArrow(self):
"""
with SCCallSiteSync(self._sc) as css:
sock_info = self._jdf.collectAsArrowToPython()
- return list(_load_from_socket(sock_info, ArrowSerializer()))
+ return list(_load_from_socket(sock_info, ArrowStreamSerializer()))
--- End diff --
We also need to update the description of `_collectAsArrow()`?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
