Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/6782#discussion_r33526479
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeFunctions.scala
---
@@ -0,0 +1,176 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.sql.catalyst.expressions
+
+import org.apache.spark.sql.catalyst.analysis.TypeCheckResult
+import
org.apache.spark.sql.catalyst.expressions.codegen.{GeneratedExpressionCode,
CodeGenContext}
+import org.apache.spark.sql.catalyst.util.DateUtils
+import org.apache.spark.sql.types._
+import org.apache.spark.unsafe.types.UTF8String
+
+/**
+ * Adds a number of days to startdate: date_add('2008-12-31', 1) =
'2009-01-01'.
+ */
+case class DateAdd(startDate: Expression, days: Expression) extends
Expression {
+ override def children: Seq[Expression] = startDate :: days :: Nil
+
+ override def foldable: Boolean = startDate.foldable && days.foldable
+ override def nullable: Boolean = startDate.nullable || days.nullable
+
+ override def checkInputDataTypes(): TypeCheckResult = {
+ val supportedLeftType = Seq(StringType, DateType, TimestampType,
NullType)
+ if (!supportedLeftType.contains(startDate.dataType)) {
+ TypeCheckResult.TypeCheckFailure(
+ s"type of startdate expression in DateAdd should be
string/timestamp/date," +
+ s" not ${startDate.dataType}")
+ } else if (days.dataType != IntegerType && days.dataType != NullType) {
+ TypeCheckResult.TypeCheckFailure(
+ s"type of days expression in DateAdd should be int, not
${days.dataType}.")
+ } else {
+ TypeCheckResult.TypeCheckSuccess
--- End diff --
@davies agree that 100% compatibility is not a goal and mysql's behavior
seems reasonable here. It would be good though to more holistically design our
data handling API and make sure we aren't missing anything. For example, today
we can handle queries like: `WHERE timestamp > "2014-01"` but only because we
convert the timestamp to a string and the strings compare lexicographically as
you would hope.
However, this feels like a hack to me and it would be much more efficient
if we could avoid creating strings for every date and instead could covert
"2014-01" into an `Int` internally. How does mysql handle partial dates like
this?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
