Ritesh17rb opened a new pull request, #392:
URL: https://github.com/apache/datafusion-spark/pull/392

   ## Which issue does this PR close?
   
   Closes #235.
   
   ## Rationale for this change
   
   Spark's `timestamp_millis` accepts integral input and converts milliseconds 
to microseconds with checked multiplication. DataFusion's `to_timestamp_millis` 
accepts additional types and stores milliseconds, giving it a different range 
and overflow behavior.
   
   ## What changes are included in this PR?
   
   - Implement and register `timestamp_millis` with scalar and array execution, 
microsecond precision, session-time-zone metadata, and input nullability.
   - Reuse `SparkInputTypes::check_only` from #381 and its lossless 
integer-widening policy. Accept Arrow UInt8/UInt16/UInt32 and reject UInt64.
   - Raise `long overflow` outside the valid millisecond range in both ANSI 
modes. Add SQL/unit coverage and regenerate the function reference.
   
   ## Are these changes tested?
   
   Formatting, workspace Clippy with all targets/features, 594 unit tests, 3 
documentation tests, all 516 SQL test runs, and documentation generation pass. 
Two existing documentation examples are ignored.
   
   The changed SQL file passes in both execution modes. The PySpark runner 
matches all 38 comparable records against Spark 4.2.0; four Arrow-specific 
records are checked locally. Coverage includes both overflow boundaries, 
negative values, NULLs, integer widths, dictionaries, rejected types, time 
zones, and DST.
   
   Detailed validation output and reproduction instructions are provided in a 
PR comment.
   
   ## Are there any user-facing changes?
   
   Spark mode now provides `timestamp_millis(milliseconds)`. For example, 
`timestamp_millis(-1::BIGINT)` returns one millisecond before the UTC epoch.
   
   Surfaced by the audit-datafusion-spark-expression skill.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to