Github user ueshin commented on the issue:
https://github.com/apache/spark/pull/19607
I have no idea about the reason but seems like there is a difference of
handling DST in `spark.createDataFrame(data, schema=schema)` between Jenkins
and my local environments.
The debug print by `df.show()` from the test
[tests.py#L3487](https://github.com/ueshin/apache-spark/blob/9101a3a12f17b5bd633756139eaa2cb3ee9bb33c/python/pyspark/sql/tests.py#L3487)
was:
```
+---+-------------------+
|idx| timestamp|
+---+-------------------+
| 0|1969-01-01 01:01:01|
| 1|2012-02-02 02:02:02|
| 2| null|
| 3|2100-04-04 04:04:04|
+---+-------------------+
```
but in my local:
```
+---+-------------------+
|idx| timestamp|
+---+-------------------+
| 0|1969-01-01 01:01:01|
| 1|2012-02-02 02:02:02|
| 2| null|
| 3|2100-04-04 05:04:04|
+---+-------------------+
```
Could you please let me know if I miss something or you have any ideas?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
