Github user maver1ck commented on a diff in the pull request:
https://github.com/apache/spark/pull/19246#discussion_r139291502
--- Diff: python/pyspark/sql/types.py ---
@@ -410,6 +410,24 @@ def __init__(self, name, dataType, nullable=True,
metadata=None):
self.dataType = dataType
self.nullable = nullable
self.metadata = metadata or {}
+ self.needConversion = dataType.needConversion
+ self.toInternal = dataType.toInternal
+ self.fromInternal = dataType.fromInternal
+
+ def __getstate__(self):
--- End diff --
We need to handle pickle by ourselves because we have fields with function
values
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
