HyukjinKwon commented on a change in pull request #34401:
URL: https://github.com/apache/spark/pull/34401#discussion_r753876468
##########
File path: python/pyspark/sql/tests/test_arrow.py
##########
@@ -205,6 +205,49 @@ def test_toPandas_fallback_disabled(self):
with self.assertRaisesRegex(Exception, "Unsupported type"):
df.toPandas()
+ def test_toPandas_empty_df_arrow_enabled(self):
+ # SPARK-30537 test that toPandas() on an empty dataframe has the
correct dtypes
+ # when arrow is enabled
+ from datetime import datetime, date
+ from decimal import Decimal
+
+ schema = StructType(
+ [
+ StructField("a", StringType(), True),
+ StructField("a", IntegerType(), True),
+ StructField("c", TimestampType(), True),
Review comment:
We will probably also have to add `TimestampNTZ` and
`DayTimeIntervalType` to test. `DayTimeIntervalType` might have an issue (see
also https://github.com/apache/spark/pull/34631/files#r751947212)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
