Github user icexelloss commented on a diff in the pull request:
https://github.com/apache/spark/pull/20531#discussion_r166649880
--- Diff: python/pyspark/sql/tests.py ---
@@ -4562,14 +4585,14 @@ def test_basic(self):
self.assertPandasEqual(expected4.toPandas(), result4.toPandas())
def test_unsupported_types(self):
- from pyspark.sql.types import ArrayType, DoubleType, MapType
+ from pyspark.sql.types import DoubleType, MapType
from pyspark.sql.functions import pandas_udf, PandasUDFType
with QuietTest(self.sc):
with self.assertRaisesRegexp(NotImplementedError, 'not
supported'):
- @pandas_udf(ArrayType(DoubleType()),
PandasUDFType.GROUPED_AGG)
+ @pandas_udf(ArrayType(ArrayType(TimestampType())),
PandasUDFType.GROUPED_AGG)
--- End diff --
Why is `ArrayType(TimestampType())` a special case that is not supported?
(I haven't fully tested this when implementing this feature, is only array of
primitives supported?)
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
