Ritesh17rb opened a new pull request, #392: URL: https://github.com/apache/datafusion-spark/pull/392
## Which issue does this PR close? Closes #235. ## Rationale for this change Spark's `timestamp_millis` accepts integral input and converts milliseconds to microseconds with checked multiplication. DataFusion's `to_timestamp_millis` accepts additional types and stores milliseconds, giving it a different range and overflow behavior. ## What changes are included in this PR? - Implement and register `timestamp_millis` with scalar and array execution, microsecond precision, session-time-zone metadata, and input nullability. - Reuse `SparkInputTypes::check_only` from #381 and its lossless integer-widening policy. Accept Arrow UInt8/UInt16/UInt32 and reject UInt64. - Raise `long overflow` outside the valid millisecond range in both ANSI modes. Add SQL/unit coverage and regenerate the function reference. ## Are these changes tested? Formatting, workspace Clippy with all targets/features, 594 unit tests, 3 documentation tests, all 516 SQL test runs, and documentation generation pass. Two existing documentation examples are ignored. The changed SQL file passes in both execution modes. The PySpark runner matches all 38 comparable records against Spark 4.2.0; four Arrow-specific records are checked locally. Coverage includes both overflow boundaries, negative values, NULLs, integer widths, dictionaries, rejected types, time zones, and DST. Detailed validation output and reproduction instructions are provided in a PR comment. ## Are there any user-facing changes? Spark mode now provides `timestamp_millis(milliseconds)`. For example, `timestamp_millis(-1::BIGINT)` returns one millisecond before the UTC epoch. Surfaced by the audit-datafusion-spark-expression skill. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
