Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/13912#discussion_r71127773
--- Diff: python/pyspark/sql/readwriter.py ---
@@ -328,6 +328,10 @@ def csv(self, path, schema=None, sep=None,
encoding=None, quote=None, escape=Non
applies to both date type and timestamp type.
By default, it is None
which means trying to parse times and date by
``java.sql.Timestamp.valueOf()`` and
``java.sql.Date.valueOf()``.
+ :param timezone: defines the timezone to be used for both date
type and timestamp type.
+ If a timezone is specified in the data, this will
load them after
--- End diff --
I thought it loses the timezone information after being loaded into Spark.
I mean, `Timestamp` and `Date` instances don't have timezone information in
them. The timezone specified in the input is being used in the example..
I am sorry that I think I didn't understand cleanly. Do you mind if I ask
what you expect in before being read, after being read (in dataframe) and after
being written?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
