alamb commented on code in PR #9077: URL: https://github.com/apache/arrow-datafusion/pull/9077#discussion_r1510295368
########## datafusion/functions/src/datetime/to_unixtime.rs: ########## @@ -0,0 +1,85 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +use std::any::Any; + +use arrow::datatypes::DataType; + +use crate::datetime::common::*; +use datafusion_common::{exec_err, Result}; +use datafusion_expr::{ColumnarValue, ScalarUDFImpl, Signature, Volatility}; + +use super::to_timestamp::ToTimestampSecondsFunc; + +#[derive(Debug)] +pub(super) struct ToUnixtimeFunc { + signature: Signature, +} + +impl ToUnixtimeFunc { + pub fn new() -> Self { + Self { + signature: Signature::variadic_any(Volatility::Immutable), + } + } +} + +impl ScalarUDFImpl for ToUnixtimeFunc { + fn as_any(&self) -> &dyn Any { + self + } + + fn name(&self) -> &str { + "to_unixtime" + } + + fn signature(&self) -> &Signature { + &self.signature + } + + fn return_type(&self, _arg_types: &[DataType]) -> Result<DataType> { + Ok(DataType::Int64) + } + + fn invoke(&self, args: &[ColumnarValue]) -> Result<ColumnarValue> { + if args.is_empty() { + return exec_err!("to_date function requires 1 or more arguments, got 0"); + } + + // validate that any args after the first one are Utf8 + if args.len() > 1 { + if let Some(value) = validate_data_types(args, "to_unixtime") { + return value; + } + } + + match args[0].data_type() { + DataType::Int32 + | DataType::Int64 + | DataType::Null + | DataType::Float64 + | DataType::Date32 + | DataType::Date64 => args[0].cast_to(&DataType::Int64, None), Review Comment: These types look a little strange to me. Why is Int32/Int64/Float supported? If that is intentional, can you please add tests of calling `to_unixtime` with arguments of those types? Also, the spark to_unixtime function https://spark.apache.org/docs/2.3.0/api/sql/index.html#to_unix_timestamp seems to also accept timestamps (`DateTime::Timestamp(..)`) -- should this function do so as well? Also I think casting a `Date32` to an `Int64` will yield the number of seconds https://docs.rs/arrow/latest/arrow/datatypes/enum.DataType.html#variant.Date32 However, casting a `Date64` to an `Int64` I think will yield the number of *milliseconds* since the unix epoch https://docs.rs/arrow/latest/arrow/datatypes/enum.DataType.html#variant.Date64 which doesn't sound correct. Perhaps you can write a test like the following to be sure: ```sql select to_unixtime(arrow_cast('2020-09-08T12:00:00+00:00', 'Date64')); ``` ########## datafusion-examples/examples/to_unixtime.rs: ########## @@ -0,0 +1,60 @@ +// Licensed to the Apache Software Foundation (ASF) under one Review Comment: FWIW I don't think we need to make a to_unixtime example -- I think the other examples for using the functions are plenty thorough. In fact I recommend we remove this example so it doesn't become a pattern for all newly added datetime functions ( could do this as a follow on PR ) ########## datafusion/functions/src/datetime/to_unixtime.rs: ########## @@ -0,0 +1,85 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +use std::any::Any; + +use arrow::datatypes::DataType; + +use crate::datetime::common::*; +use datafusion_common::{exec_err, Result}; +use datafusion_expr::{ColumnarValue, ScalarUDFImpl, Signature, Volatility}; + +use super::to_timestamp::ToTimestampSecondsFunc; + +#[derive(Debug)] +pub(super) struct ToUnixtimeFunc { + signature: Signature, +} + +impl ToUnixtimeFunc { + pub fn new() -> Self { + Self { + signature: Signature::variadic_any(Volatility::Immutable), + } + } +} + +impl ScalarUDFImpl for ToUnixtimeFunc { + fn as_any(&self) -> &dyn Any { + self + } + + fn name(&self) -> &str { + "to_unixtime" + } + + fn signature(&self) -> &Signature { + &self.signature + } + + fn return_type(&self, _arg_types: &[DataType]) -> Result<DataType> { + Ok(DataType::Int64) + } + + fn invoke(&self, args: &[ColumnarValue]) -> Result<ColumnarValue> { + if args.is_empty() { + return exec_err!("to_date function requires 1 or more arguments, got 0"); Review Comment: ```suggestion return exec_err!("to_unixtime function requires 1 or more arguments, got 0"); ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
