Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/19459#discussion_r149871432
--- Diff: python/pyspark/serializers.py ---
@@ -213,7 +213,15 @@ def __repr__(self):
return "ArrowSerializer"
-def _create_batch(series):
+def _create_batch(series, copy=False):
--- End diff --
Hmm, I guess it depends.
With the method, it can reduce the number of copy if `s` doesn't include
null values, but also it might increase the number if `s` includes null values
and `copy=True`.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
