[
https://issues.apache.org/jira/browse/ARROW-3702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16676110#comment-16676110
]
Javier Luraschi commented on ARROW-3702:
----------------------------------------
[~romainfrancois] Actually, cross posting feedback from Spark PR... This is a
strong argument for mapping POSIXct to Timestamp since the Arrow flatbuffer's
file does mention that a date MUST be divisible by 86400000, which I believe is
currently not the case:
[https://github.com/apache/arrow/blob/master/format/Schema.fbs]
{code:java}
/// Date is either a 32-bit or 64-bit type representing elapsed time since UNIX
/// epoch (1970-01-01), stored in either of two units:
///
/// * Milliseconds (64 bits) indicating UNIX time elapsed since the epoch (no
/// leap seconds), where the values are evenly divisible by 86400000
/// * Days (32 bits) since the UNIX epoch{code}
> [R] POSIXct mapped to DateType not TimestampType?
> -------------------------------------------------
>
> Key: ARROW-3702
> URL: https://issues.apache.org/jira/browse/ARROW-3702
> Project: Apache Arrow
> Issue Type: Improvement
> Components: R
> Reporter: Javier Luraschi
> Priority: Major
>
> Why was POSIXct mapped to
> [DataType|https://arrow.apache.org/docs/cpp/classarrow_1_1_date_type.html#a6aea1fcfd9f998e8fa50f5ae62dbd7e6]
> not
> [TimestampType|https://arrow.apache.org/docs/cpp/classarrow_1_1_timestamp_type.html#a88e0ba47b82571b3fc3798b6c099499b]?
> What are the PRO/CONs from each approach?
> This is mostly to interoperate with Spark which choose to map POSIXct to
> Timestamps since in Spark, not Arrow, dates do not have a time component.
> There is a way to make this work in Spark with POSIXct mapped to DateType by
> mapping DateType to timestamps, so mostly looking to understand tradeoffs.
> One particular question, timestamps in arrow seem to support timezones,
> wouldn't it make more sense to map POSIXct to timestamps?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
