Github user BryanCutler commented on a diff in the pull request:
https://github.com/apache/spark/pull/19646#discussion_r149192441
--- Diff: python/pyspark/sql/tests.py ---
@@ -2592,6 +2592,16 @@ def test_create_dataframe_from_array_of_long(self):
df = self.spark.createDataFrame(data)
self.assertEqual(df.first(), Row(longarray=[-9223372036854775808,
0, 9223372036854775807]))
+ @unittest.skipIf(not _have_pandas, "Pandas not installed")
+ def test_create_dataframe_from_pandas_with_timestamp(self):
+ import pandas as pd
+ from datetime import datetime
+ pdf = pd.DataFrame({"ts": [datetime(2017, 10, 31, 1, 1, 1)],
+ "d": [pd.Timestamp.now().date()]})
+ df = self.spark.createDataFrame(pdf)
--- End diff --
Yes, looks like that needs to be fixed also. I thought it was working when
schema was supplied, but I'll double-check and add that into the tests.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
