Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/13912#discussion_r71126016
--- Diff: python/pyspark/sql/readwriter.py ---
@@ -328,6 +328,10 @@ def csv(self, path, schema=None, sep=None,
encoding=None, quote=None, escape=Non
applies to both date type and timestamp type.
By default, it is None
which means trying to parse times and date by
``java.sql.Timestamp.valueOf()`` and
``java.sql.Date.valueOf()``.
+ :param timezone: defines the timezone to be used for both date
type and timestamp type.
+ If a timezone is specified in the data, this will
load them after
--- End diff --
In the example above, the input specifies a timezone and that must be used
to interpret it. You say "it becomes in the dataframe" but what it becomes is a
timestamp, which is unambiguous and has no timezone. Timezone matters when
converting back to a string for display, but, your example only shows the
parameter used on reading, and shows no timezone in the output. I am not sure
that this is the intended behavior?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
