[
https://issues.apache.org/jira/browse/SPARK-21940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16160936#comment-16160936
]
Yanbo Liang commented on SPARK-21940:
-------------------------------------
[~falaki] AFAIK, Spark SQL timestamps are normalized to UTC based on available
time zone information and stored as UTC. I think R as.double(time) do the same
as Spark SQL. So any data of TimeStamp type will be interpreted according the
timezone that users operate at. If users must to interpret time to a specific
timezone, they can set their local timezone with {{Sys.setenv(TZ='GMT')}}.
However, if users would like to bind the timezone with timestamp, I would
recommend them to store timestamp as string and use UDF to operate them. What
do you think of it? Thanks.
> Support timezone for timestamps in SparkR
> -----------------------------------------
>
> Key: SPARK-21940
> URL: https://issues.apache.org/jira/browse/SPARK-21940
> Project: Spark
> Issue Type: Bug
> Components: SparkR
> Affects Versions: 2.2.0
> Reporter: Hossein Falaki
>
> {{SparkR::createDataFrame()}} wipes timezone attribute from POSIXct and
> POSIXlt. See following example:
> {code}
> > x <- data.frame(x = c(Sys.time()))
> > x
> x
> 1 2017-09-06 19:17:16
> > attr(x$x, "tzone") <- "Europe/Paris"
> > x
> x
> 1 2017-09-07 04:17:16
> > collect(createDataFrame(x))
> x
> 1 2017-09-06 19:17:16
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
