alamb commented on code in PR #20674:
URL: https://github.com/apache/datafusion/pull/20674#discussion_r2879269170
##########
datafusion/functions/src/datetime/date_part.rs:
##########
@@ -321,6 +323,12 @@ fn is_epoch(part: &str) -> bool {
matches!(part.to_lowercase().as_str(), "epoch")
}
+fn is_nanosecond(part: &str) -> bool {
Review Comment:
Is it worth adding a note that this is required due to the fact that
nanosecond needs a different return type
##########
datafusion/functions/src/datetime/date_part.rs:
##########
@@ -517,3 +525,35 @@ fn epoch(array: &dyn Array) -> Result<ArrayRef> {
};
Ok(Arc::new(f))
}
+
+/// Invoke [`date_part`] on an `array` (e.g. Timestamp) and convert the
+/// result to a total number of nanoseconds as an Int64 array.
+fn seconds_ns(array: &dyn Array) -> Result<ArrayRef> {
+ let secs = date_part(array, DatePart::Second)?;
+ // This assumes array is primitive and not a dictionary
+ let secs = as_int32_array(secs.as_ref())?;
+ let subsecs = date_part(array, DatePart::Nanosecond)?;
+ let subsecs = as_int32_array(subsecs.as_ref())?;
+
+ // Special case where there are no nulls.
+ if subsecs.null_count() == 0 {
Review Comment:
Would it be safer to also check sec.nulls too? It does feel like any nulls
in subsecs would also be in secs and vica versa
##########
datafusion/sqllogictest/test_files/datetime/date_part.slt:
##########
@@ -484,9 +504,10 @@ SELECT EXTRACT('microsecond' FROM timestamp
'2020-09-08T12:00:12.12345678+00:00'
----
12123456
-query error DataFusion error: This feature is not implemented: Date part
Nanosecond not supported
+query I
Review Comment:
nice
##########
datafusion/functions/src/datetime/date_part.rs:
##########
@@ -517,3 +525,35 @@ fn epoch(array: &dyn Array) -> Result<ArrayRef> {
};
Ok(Arc::new(f))
}
+
+/// Invoke [`date_part`] on an `array` (e.g. Timestamp) and convert the
+/// result to a total number of nanoseconds as an Int64 array.
+fn seconds_ns(array: &dyn Array) -> Result<ArrayRef> {
+ let secs = date_part(array, DatePart::Second)?;
+ // This assumes array is primitive and not a dictionary
+ let secs = as_int32_array(secs.as_ref())?;
+ let subsecs = date_part(array, DatePart::Nanosecond)?;
+ let subsecs = as_int32_array(subsecs.as_ref())?;
+
+ // Special case where there are no nulls.
+ if subsecs.null_count() == 0 {
+ let r: Int64Array = binary(secs, subsecs, |secs, subsecs| {
+ (secs as i64) * 1_000_000_000 + (subsecs as i64)
+ })?;
+ Ok(Arc::new(r))
+ } else {
+ // Nulls in secs are preserved, nulls in subsecs are treated as zero
to account for the case
+ // where the number of nanoseconds overflows.
+ let r: Int64Array = secs
+ .iter()
+ .zip(subsecs)
+ .map(|(secs, subsecs)| {
+ secs.map(|secs| {
+ let subsecs = subsecs.unwrap_or(0);
Review Comment:
I didn't see any test coverage of nulls 🤔 -- maybe we can add some
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
