+1 (non-binding) On Mon, May 18, 2026 at 2:38 PM Mridul Muralidharan <[email protected]> wrote:
> > +1 > > Mridul > > On Mon, May 18, 2026 at 4:52 AM Wenchen Fan <[email protected]> wrote: > >> +1 (binding) >> >> On Mon, May 18, 2026 at 5:07 PM Menelaos Karavelas < >> [email protected]> wrote: >> >>> +1 (non-binding) >>> >>> On May 18, 2026, at 9:00 AM, serge rielau.com <[email protected]> wrote: >>> >>> +1 >>> >>> On May 15, 2026, at 6:07 PM, Stefan Kandic via dev <[email protected]> >>> wrote: >>> >>> +1 >>> >>> *From: *Max Gekk <[email protected]> >>> *Date: *Friday, 15 May 2026 at 09:53 >>> *To: *dev <[email protected]> >>> *Subject: *Re: [VOTE] SPIP: Timestamps with nanosecond precision >>> >>> +1 >>> >>> On Fri, May 15, 2026 at 9:15 AM Max Gekk <[email protected]> wrote: >>> >>> Hi Spark devs, >>> >>> I would like to call a vote on adopting the following SPIP as an >>> official Spark SPIP, following the discussion on the list. >>> Motivation >>> Spark SQL today exposes TIMESTAMP / TIMESTAMP_NTZ / TIMESTAMP_LTZ at >>> microsecond precision. Nanosecond timestamps are increasingly common in ORC >>> and in ecosystems such as Oracle, MS SQL Server, Trino, Snowflake, and >>> DuckDB. Spark often rejects Parquet TIMESTAMP(NANOS, …) or, with legacy >>> flags, reads it as LongType, which drops timestamp semantics and time-zone >>> behavior. Users are forced to pre-normalize to micros or maintain custom >>> conversion logic. >>> >>> This SPIP aims to add first-class nanosecond-capable timestamps in SQL >>> and APIs, with a clear fractional precision parameter, while keeping a >>> practical migration path for existing microsecond workloads. >>> Proposal (summary) >>> >>> - SQL surface: TIMESTAMP_NTZ(n), TIMESTAMP_LTZ(n), and TIMESTAMP(n) >>> (including WITHOUT TIME ZONE / WITH LOCAL TIME ZONE spellings), with >>> optional n in [0, 9]. The scope for the initial milestone focuses on 7 >>> <= n >>> <= 9; unparameterized types keep today’s defaults. >>> - New Catalyst types: additive parameterized types (e.g. >>> TimestampNTZNanosType(n) / companion for LTZ) rather than silently >>> widening >>> existing singleton types. >>> - Logical value model: epochMicros (long) + nanosOfMicro in [0, 999] >>> (short) with a single normalization rule, so Spark continues to build on >>> the existing microsecond datetime “currency” while adding a bounded >>> sub-micro correction. >>> - Scope boundaries: explicit non-goals (e.g. not redefining NTZ/LTZ >>> session semantics, not promising every connector end-to-end in v1), >>> risks/mitigations, rough ~25 person-week estimate and success “exams” are >>> documented in the SPIP and JIRA. >>> >>> Full detail, API tables, rejected alternatives, and test/migration >>> criteria are in the SPIP document and JIRA description. >>> >>> Relevant links >>> >>> - SPIP (Google Doc): >>> >>> https://docs.google.com/document/d/1DeW15QueI4PdRyPm6C6jsTZFmIjbXX2j4h-Ja5W_fsg/edit?usp=sharing >>> - Discussion thread: >>> https://lists.apache.org/thread/xvv4qt9dpnb1kszxdqlxyv9b46749ypo >>> - JIRA: https://issues.apache.org/jira/browse/SPARK-56822 >>> >>> Vote >>> Please vote on accepting this proposal as an official SPIP (the SPIP >>> text above; implementation and follow-up JIRAs can land incrementally after >>> acceptance). >>> [ ] +1: Accept the proposal as an official SPIP >>> [ ] +0: No opinion >>> [ ] -1: I do not think we should adopt this SPIP (please explain why) >>> The vote will remain open for at least 72 hours. >>> >>> Thanks to everyone who commented on the discussion thread and helped >>> refine the design. >>> >>> Best regards, >>> Max Gekk >>> >>> >>> >>>
