GitHub user holdenk opened a pull request:
https://github.com/apache/spark/pull/14398
[SPARK-16774][SQL] Fix use of deprecated timestamp constructor & improve
timezone handling
## What changes were proposed in this pull request?
Removes the deprecated timestamp constructor and incidentally fixes the use
which was using system timezone rather than the one specified when working near
DST.
This change also causes the roundtrip tests to fail since it now actually
uses all the timezones near DST boundaries where it didn't before.
Note: this is only a partial the solution, longer term we should follow up
with https://issues.apache.org/jira/browse/SPARK-16788 to avoid this problem &
simplify our timezone handling code.
## How was this patch tested?
New tests for two timezones added so even if user timezone happens to
coincided with one, the other tests should still fail. Important note: this
(temporarily) disables the round trip tests until we can fix the issue more
thoroughly.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/holdenk/spark
SPARK-16774-fix-use-of-deprecated-timestamp-constructor
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/14398.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #14398
----
commit 7c0fd8d4f50378b419bfd0059fe31f6e95205fb3
Author: Holden Karau <[email protected]>
Date: 2016-07-28T04:23:00Z
Stop using deprecated Timestamp constructor
commit 634a05e3c7afed2987efdcb0680b2e3547ae10a2
Author: Holden Karau <[email protected]>
Date: 2016-07-29T00:25:45Z
Wait the old timezone code was badnews bears - did stuff based implicitly
on local system timezone - lets avoid that
commit 6233b4cbdc55930334634b9acadea627117e6e23
Author: Holden Karau <[email protected]>
Date: 2016-07-29T05:55:13Z
Simplify & disable the roundtrip test - it wasn't really running anyways
since it was always falling back to system timezone in the fall through case
commit e4508e32662f9eba4621c7ffb54ec7661dbc54fd
Author: Holden Karau <[email protected]>
Date: 2016-07-29T06:01:21Z
not used anymore
commit 0385d8bd3f7ea85abb14191a4f3eb7eb1dbd4582
Author: Holden Karau <[email protected]>
Date: 2016-07-29T06:07:37Z
Just remove the roundtrip test for now - can be added back from git history
later
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
