Martin Grund created SPARK-42027:
------------------------------------
Summary: CreateDataframe from Pandas with Struct and Timestamp
Key: SPARK-42027
URL: https://issues.apache.org/jira/browse/SPARK-42027
Project: Spark
Issue Type: Sub-task
Components: Connect
Affects Versions: 3.4.0
Reporter: Martin Grund
The following should be supported and correctly truncate the nanosecond
timestamps.
{code:python}
from datetime import datetime, timezone, timedelta
from pandas import Timestamp
ts=Timestamp(year=2019, month=1, day=1, nanosecond=500,
tz=timezone(timedelta(hours=-8)))
d = pd.DataFrame({"col1": [1], "col2": [{"a":1, "b":2.32, "c":ts}]})
spark.createDataFrame(d).collect()
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
