[GitHub] spark pull request #16476: [SPARK-19084][SQL][WIP] Implement expression fiel...
Github user gczsjdy commented on a diff in the pull request:
https://github.com/apache/spark/pull/16476#discussion_r95294460
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/conditionalExpressions.scala
---
@@ -340,3 +344,102 @@ object CaseKeyWhen {
CaseWhen(cases, elseValue)
}
}
+
+/**
+ * A function that returns the index of expr in (expr1, expr2, ...) list
or 0 if not found.
+ * It takes at least 2 parameters, and all parameters should be subtype of
AtomicType or NullType.
+ * It's also acceptable to give parameters of different types.
+ * If the search string is NULL, the return value is 0 because NULL fails
equality comparison with any value.
+ * When the paramters have different types, comparing will be done based
on type firstly,
+ * for example, ''999'' won't be considered equal with 999, no implicit
cast will be done here.
+ */
+@ExpressionDescription(
+ usage = "_FUNC_(expr, expr1, expr2, ...) - Returns the index of expr in
the expr1, expr2, ... or 0 if not found.",
+ extended = """
+Examples:
+ > SELECT _FUNC_(10, 9, 3, 10, 4);
+ 3
+ > SELECT _FUNC_('a', 'b', 'c', 'd', 'a');
+ 4
+ > SELECT _FUNC_('999', 'a', 999, 9.99, '999');
+ 4
+ """)
+case class Field(children: Seq[Expression]) extends Expression {
+
+ /** Even if expr is not found in (expr1, expr2, ...) list, the value
will be 0, not null */
+ override def nullable: Boolean = false
+ override def foldable: Boolean = children.forall(_.foldable)
+
+ private lazy val ordering =
TypeUtils.getInterpretedOrdering(children(0).dataType)
+
+ private val dataTypeMatchIndex: Seq[Int] = children.tail.zip(Stream from
1).filter(
--- End diff --
Actually, it's short for `zipWithIndex.map(x => (x._1, x._2 + 1))`.
I realized it makes people confused, and have changed it.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #16476: [SPARK-19084][SQL][WIP] Implement expression fiel...
Github user gczsjdy commented on a diff in the pull request:
https://github.com/apache/spark/pull/16476#discussion_r95294259
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/conditionalExpressions.scala
---
@@ -340,3 +344,102 @@ object CaseKeyWhen {
CaseWhen(cases, elseValue)
}
}
+
+/**
+ * A function that returns the index of expr in (expr1, expr2, ...) list
or 0 if not found.
+ * It takes at least 2 parameters, and all parameters should be subtype of
AtomicType or NullType.
+ * It's also acceptable to give parameters of different types.
+ * If the search string is NULL, the return value is 0 because NULL fails
equality comparison with any value.
+ * When the paramters have different types, comparing will be done based
on type firstly,
+ * for example, ''999'' won't be considered equal with 999, no implicit
cast will be done here.
+ */
+@ExpressionDescription(
+ usage = "_FUNC_(expr, expr1, expr2, ...) - Returns the index of expr in
the expr1, expr2, ... or 0 if not found.",
+ extended = """
+Examples:
+ > SELECT _FUNC_(10, 9, 3, 10, 4);
+ 3
+ > SELECT _FUNC_('a', 'b', 'c', 'd', 'a');
+ 4
+ > SELECT _FUNC_('999', 'a', 999, 9.99, '999');
+ 4
+ """)
+case class Field(children: Seq[Expression]) extends Expression {
+
+ /** Even if expr is not found in (expr1, expr2, ...) list, the value
will be 0, not null */
+ override def nullable: Boolean = false
+ override def foldable: Boolean = children.forall(_.foldable)
+
+ private lazy val ordering =
TypeUtils.getInterpretedOrdering(children(0).dataType)
+
+ private val dataTypeMatchIndex: Seq[Int] = children.tail.zip(Stream from
1).filter(
+_._1.dataType == children.head.dataType).map(_._2)
+
+ override def checkInputDataTypes(): TypeCheckResult = {
+if (children.length <= 1) {
+ TypeCheckResult.TypeCheckFailure(s"FIELD requires at least 2
arguments")
+} else if (!children.forall(
+e => e.dataType.isInstanceOf[AtomicType] ||
e.dataType.isInstanceOf[NullType])) {
+ TypeCheckResult.TypeCheckFailure(s"FIELD requires all arguments to
be of AtomicType")
+} else
+ TypeCheckResult.TypeCheckSuccess
+ }
+
+ override def dataType: DataType = IntegerType
+ override def eval(input: InternalRow): Any = {
+val target = children.head.eval(input)
+val targetDataType = children.head.dataType
--- End diff --
Got it
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #16476: [SPARK-19084][SQL][WIP] Implement expression fiel...
Github user gczsjdy commented on a diff in the pull request:
https://github.com/apache/spark/pull/16476#discussion_r95292505
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/conditionalExpressions.scala
---
@@ -340,3 +344,102 @@ object CaseKeyWhen {
CaseWhen(cases, elseValue)
}
}
+
+/**
+ * A function that returns the index of expr in (expr1, expr2, ...) list
or 0 if not found.
+ * It takes at least 2 parameters, and all parameters should be subtype of
AtomicType or NullType.
+ * It's also acceptable to give parameters of different types.
+ * If the search string is NULL, the return value is 0 because NULL fails
equality comparison with any value.
+ * When the paramters have different types, comparing will be done based
on type firstly,
+ * for example, ''999'' won't be considered equal with 999, no implicit
cast will be done here.
+ */
+@ExpressionDescription(
+ usage = "_FUNC_(expr, expr1, expr2, ...) - Returns the index of expr in
the expr1, expr2, ... or 0 if not found.",
+ extended = """
+Examples:
+ > SELECT _FUNC_(10, 9, 3, 10, 4);
+ 3
+ > SELECT _FUNC_('a', 'b', 'c', 'd', 'a');
+ 4
+ > SELECT _FUNC_('999', 'a', 999, 9.99, '999');
+ 4
+ """)
+case class Field(children: Seq[Expression]) extends Expression {
+
+ /** Even if expr is not found in (expr1, expr2, ...) list, the value
will be 0, not null */
+ override def nullable: Boolean = false
+ override def foldable: Boolean = children.forall(_.foldable)
+
+ private lazy val ordering =
TypeUtils.getInterpretedOrdering(children(0).dataType)
+
+ private val dataTypeMatchIndex: Seq[Int] = children.tail.zip(Stream from
1).filter(
+_._1.dataType == children.head.dataType).map(_._2)
--- End diff --
Got it, thx.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #16476: [SPARK-19084][SQL][WIP] Implement expression fiel...
Github user gczsjdy commented on a diff in the pull request:
https://github.com/apache/spark/pull/16476#discussion_r95292481
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/conditionalExpressions.scala
---
@@ -340,3 +344,102 @@ object CaseKeyWhen {
CaseWhen(cases, elseValue)
}
}
+
+/**
+ * A function that returns the index of expr in (expr1, expr2, ...) list
or 0 if not found.
+ * It takes at least 2 parameters, and all parameters should be subtype of
AtomicType or NullType.
+ * It's also acceptable to give parameters of different types.
+ * If the search string is NULL, the return value is 0 because NULL fails
equality comparison with any value.
+ * When the paramters have different types, comparing will be done based
on type firstly,
+ * for example, ''999'' won't be considered equal with 999, no implicit
cast will be done here.
+ */
+@ExpressionDescription(
+ usage = "_FUNC_(expr, expr1, expr2, ...) - Returns the index of expr in
the expr1, expr2, ... or 0 if not found.",
+ extended = """
+Examples:
+ > SELECT _FUNC_(10, 9, 3, 10, 4);
+ 3
+ > SELECT _FUNC_('a', 'b', 'c', 'd', 'a');
+ 4
+ > SELECT _FUNC_('999', 'a', 999, 9.99, '999');
+ 4
+ """)
+case class Field(children: Seq[Expression]) extends Expression {
+
+ /** Even if expr is not found in (expr1, expr2, ...) list, the value
will be 0, not null */
+ override def nullable: Boolean = false
+ override def foldable: Boolean = children.forall(_.foldable)
+
+ private lazy val ordering =
TypeUtils.getInterpretedOrdering(children(0).dataType)
+
+ private val dataTypeMatchIndex: Seq[Int] = children.tail.zip(Stream from
1).filter(
--- End diff --
Won't it be inconsistent with children? Since children's type is
Seq[Expression]
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #16476: [SPARK-19084][SQL][WIP] Implement expression fiel...
Github user chenghao-intel commented on a diff in the pull request:
https://github.com/apache/spark/pull/16476#discussion_r95283107
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala ---
@@ -1528,6 +1528,18 @@ object functions {
def factorial(e: Column): Column = withExpr { Factorial(e.expr) }
/**
+* Returns the index of str in (str1, str2, ...) list or 0 if not found.
+* It takes at least 2 parameters, and all parameters' types should be
subtypes of AtomicType.
+*
+* @group normal_funcs
+* @since 2.2.0
+*/
+ @scala.annotation.varargs
+ def field(expr1: Column, expr2: Column, exprs: Column*): Column =
withExpr {
--- End diff --
@rxin can you explain a little bit why we remove this?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #16476: [SPARK-19084][SQL][WIP] Implement expression fiel...
Github user chenghao-intel commented on a diff in the pull request:
https://github.com/apache/spark/pull/16476#discussion_r95282681
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/conditionalExpressions.scala
---
@@ -340,3 +344,102 @@ object CaseKeyWhen {
CaseWhen(cases, elseValue)
}
}
+
+/**
+ * A function that returns the index of expr in (expr1, expr2, ...) list
or 0 if not found.
+ * It takes at least 2 parameters, and all parameters should be subtype of
AtomicType or NullType.
+ * It's also acceptable to give parameters of different types.
+ * If the search string is NULL, the return value is 0 because NULL fails
equality comparison with any value.
+ * When the paramters have different types, comparing will be done based
on type firstly,
+ * for example, ''999'' won't be considered equal with 999, no implicit
cast will be done here.
+ */
+@ExpressionDescription(
+ usage = "_FUNC_(expr, expr1, expr2, ...) - Returns the index of expr in
the expr1, expr2, ... or 0 if not found.",
+ extended = """
+Examples:
+ > SELECT _FUNC_(10, 9, 3, 10, 4);
+ 3
+ > SELECT _FUNC_('a', 'b', 'c', 'd', 'a');
+ 4
+ > SELECT _FUNC_('999', 'a', 999, 9.99, '999');
+ 4
+ """)
+case class Field(children: Seq[Expression]) extends Expression {
+
+ /** Even if expr is not found in (expr1, expr2, ...) list, the value
will be 0, not null */
+ override def nullable: Boolean = false
+ override def foldable: Boolean = children.forall(_.foldable)
+
+ private lazy val ordering =
TypeUtils.getInterpretedOrdering(children(0).dataType)
+
+ private val dataTypeMatchIndex: Seq[Int] = children.tail.zip(Stream from
1).filter(
+_._1.dataType == children.head.dataType).map(_._2)
+
+ override def checkInputDataTypes(): TypeCheckResult = {
+if (children.length <= 1) {
+ TypeCheckResult.TypeCheckFailure(s"FIELD requires at least 2
arguments")
+} else if (!children.forall(
+e => e.dataType.isInstanceOf[AtomicType] ||
e.dataType.isInstanceOf[NullType])) {
+ TypeCheckResult.TypeCheckFailure(s"FIELD requires all arguments to
be of AtomicType")
+} else
+ TypeCheckResult.TypeCheckSuccess
+ }
+
+ override def dataType: DataType = IntegerType
+ override def eval(input: InternalRow): Any = {
+val target = children.head.eval(input)
+val targetDataType = children.head.dataType
+@tailrec def findEqual(index: Int): Int = {
+ if (index == dataTypeMatchIndex.size) {
+0
+ } else {
+val value = children(dataTypeMatchIndex(index)).eval(input)
+if (value != null && ordering.equiv(target, value))
--- End diff --
use braces. see https://github.com/databricks/scala-style-guide#curly
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #16476: [SPARK-19084][SQL][WIP] Implement expression fiel...
Github user chenghao-intel commented on a diff in the pull request:
https://github.com/apache/spark/pull/16476#discussion_r95282465
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/conditionalExpressions.scala
---
@@ -340,3 +344,102 @@ object CaseKeyWhen {
CaseWhen(cases, elseValue)
}
}
+
+/**
+ * A function that returns the index of expr in (expr1, expr2, ...) list
or 0 if not found.
+ * It takes at least 2 parameters, and all parameters should be subtype of
AtomicType or NullType.
+ * It's also acceptable to give parameters of different types.
+ * If the search string is NULL, the return value is 0 because NULL fails
equality comparison with any value.
+ * When the paramters have different types, comparing will be done based
on type firstly,
+ * for example, ''999'' won't be considered equal with 999, no implicit
cast will be done here.
+ */
+@ExpressionDescription(
+ usage = "_FUNC_(expr, expr1, expr2, ...) - Returns the index of expr in
the expr1, expr2, ... or 0 if not found.",
+ extended = """
+Examples:
+ > SELECT _FUNC_(10, 9, 3, 10, 4);
+ 3
+ > SELECT _FUNC_('a', 'b', 'c', 'd', 'a');
+ 4
+ > SELECT _FUNC_('999', 'a', 999, 9.99, '999');
+ 4
+ """)
+case class Field(children: Seq[Expression]) extends Expression {
+
+ /** Even if expr is not found in (expr1, expr2, ...) list, the value
will be 0, not null */
+ override def nullable: Boolean = false
+ override def foldable: Boolean = children.forall(_.foldable)
+
+ private lazy val ordering =
TypeUtils.getInterpretedOrdering(children(0).dataType)
+
+ private val dataTypeMatchIndex: Seq[Int] = children.tail.zip(Stream from
1).filter(
+_._1.dataType == children.head.dataType).map(_._2)
+
+ override def checkInputDataTypes(): TypeCheckResult = {
+if (children.length <= 1) {
+ TypeCheckResult.TypeCheckFailure(s"FIELD requires at least 2
arguments")
+} else if (!children.forall(
+e => e.dataType.isInstanceOf[AtomicType] ||
e.dataType.isInstanceOf[NullType])) {
+ TypeCheckResult.TypeCheckFailure(s"FIELD requires all arguments to
be of AtomicType")
+} else
+ TypeCheckResult.TypeCheckSuccess
+ }
+
+ override def dataType: DataType = IntegerType
+ override def eval(input: InternalRow): Any = {
+val target = children.head.eval(input)
+val targetDataType = children.head.dataType
+@tailrec def findEqual(index: Int): Int = {
+ if (index == dataTypeMatchIndex.size) {
--- End diff --
if `dataTypeMatchIndex` is `Array[Int]`, then we'd better use
`dataTypeMatchIndex.length` instead.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #16476: [SPARK-19084][SQL][WIP] Implement expression fiel...
Github user chenghao-intel commented on a diff in the pull request:
https://github.com/apache/spark/pull/16476#discussion_r95282270
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/conditionalExpressions.scala
---
@@ -340,3 +344,102 @@ object CaseKeyWhen {
CaseWhen(cases, elseValue)
}
}
+
+/**
+ * A function that returns the index of expr in (expr1, expr2, ...) list
or 0 if not found.
+ * It takes at least 2 parameters, and all parameters should be subtype of
AtomicType or NullType.
+ * It's also acceptable to give parameters of different types.
+ * If the search string is NULL, the return value is 0 because NULL fails
equality comparison with any value.
+ * When the paramters have different types, comparing will be done based
on type firstly,
+ * for example, ''999'' won't be considered equal with 999, no implicit
cast will be done here.
+ */
+@ExpressionDescription(
+ usage = "_FUNC_(expr, expr1, expr2, ...) - Returns the index of expr in
the expr1, expr2, ... or 0 if not found.",
+ extended = """
+Examples:
+ > SELECT _FUNC_(10, 9, 3, 10, 4);
+ 3
+ > SELECT _FUNC_('a', 'b', 'c', 'd', 'a');
+ 4
+ > SELECT _FUNC_('999', 'a', 999, 9.99, '999');
+ 4
+ """)
+case class Field(children: Seq[Expression]) extends Expression {
+
+ /** Even if expr is not found in (expr1, expr2, ...) list, the value
will be 0, not null */
+ override def nullable: Boolean = false
+ override def foldable: Boolean = children.forall(_.foldable)
+
+ private lazy val ordering =
TypeUtils.getInterpretedOrdering(children(0).dataType)
+
+ private val dataTypeMatchIndex: Seq[Int] = children.tail.zip(Stream from
1).filter(
+_._1.dataType == children.head.dataType).map(_._2)
+
+ override def checkInputDataTypes(): TypeCheckResult = {
+if (children.length <= 1) {
+ TypeCheckResult.TypeCheckFailure(s"FIELD requires at least 2
arguments")
+} else if (!children.forall(
+e => e.dataType.isInstanceOf[AtomicType] ||
e.dataType.isInstanceOf[NullType])) {
+ TypeCheckResult.TypeCheckFailure(s"FIELD requires all arguments to
be of AtomicType")
+} else
+ TypeCheckResult.TypeCheckSuccess
+ }
+
+ override def dataType: DataType = IntegerType
+ override def eval(input: InternalRow): Any = {
+val target = children.head.eval(input)
+val targetDataType = children.head.dataType
--- End diff --
Unused code.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #16476: [SPARK-19084][SQL][WIP] Implement expression fiel...
Github user chenghao-intel commented on a diff in the pull request:
https://github.com/apache/spark/pull/16476#discussion_r95281248
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/conditionalExpressions.scala
---
@@ -340,3 +344,102 @@ object CaseKeyWhen {
CaseWhen(cases, elseValue)
}
}
+
+/**
+ * A function that returns the index of expr in (expr1, expr2, ...) list
or 0 if not found.
+ * It takes at least 2 parameters, and all parameters should be subtype of
AtomicType or NullType.
+ * It's also acceptable to give parameters of different types.
+ * If the search string is NULL, the return value is 0 because NULL fails
equality comparison with any value.
+ * When the paramters have different types, comparing will be done based
on type firstly,
+ * for example, ''999'' won't be considered equal with 999, no implicit
cast will be done here.
+ */
+@ExpressionDescription(
+ usage = "_FUNC_(expr, expr1, expr2, ...) - Returns the index of expr in
the expr1, expr2, ... or 0 if not found.",
+ extended = """
+Examples:
+ > SELECT _FUNC_(10, 9, 3, 10, 4);
+ 3
+ > SELECT _FUNC_('a', 'b', 'c', 'd', 'a');
+ 4
+ > SELECT _FUNC_('999', 'a', 999, 9.99, '999');
+ 4
+ """)
+case class Field(children: Seq[Expression]) extends Expression {
+
+ /** Even if expr is not found in (expr1, expr2, ...) list, the value
will be 0, not null */
+ override def nullable: Boolean = false
+ override def foldable: Boolean = children.forall(_.foldable)
+
+ private lazy val ordering =
TypeUtils.getInterpretedOrdering(children(0).dataType)
+
+ private val dataTypeMatchIndex: Seq[Int] = children.tail.zip(Stream from
1).filter(
--- End diff --
`zip(Stream from 1)`, do we really need it?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #16476: [SPARK-19084][SQL][WIP] Implement expression fiel...
Github user chenghao-intel commented on a diff in the pull request:
https://github.com/apache/spark/pull/16476#discussion_r95281159
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/conditionalExpressions.scala
---
@@ -340,3 +344,102 @@ object CaseKeyWhen {
CaseWhen(cases, elseValue)
}
}
+
+/**
+ * A function that returns the index of expr in (expr1, expr2, ...) list
or 0 if not found.
+ * It takes at least 2 parameters, and all parameters should be subtype of
AtomicType or NullType.
+ * It's also acceptable to give parameters of different types.
+ * If the search string is NULL, the return value is 0 because NULL fails
equality comparison with any value.
+ * When the paramters have different types, comparing will be done based
on type firstly,
+ * for example, ''999'' won't be considered equal with 999, no implicit
cast will be done here.
+ */
+@ExpressionDescription(
+ usage = "_FUNC_(expr, expr1, expr2, ...) - Returns the index of expr in
the expr1, expr2, ... or 0 if not found.",
+ extended = """
+Examples:
+ > SELECT _FUNC_(10, 9, 3, 10, 4);
+ 3
+ > SELECT _FUNC_('a', 'b', 'c', 'd', 'a');
+ 4
+ > SELECT _FUNC_('999', 'a', 999, 9.99, '999');
+ 4
+ """)
+case class Field(children: Seq[Expression]) extends Expression {
+
+ /** Even if expr is not found in (expr1, expr2, ...) list, the value
will be 0, not null */
+ override def nullable: Boolean = false
+ override def foldable: Boolean = children.forall(_.foldable)
+
+ private lazy val ordering =
TypeUtils.getInterpretedOrdering(children(0).dataType)
+
+ private val dataTypeMatchIndex: Seq[Int] = children.tail.zip(Stream from
1).filter(
+_._1.dataType == children.head.dataType).map(_._2)
--- End diff --
`_._1.dataType.sameType(children.head.dataType)`?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #16476: [SPARK-19084][SQL][WIP] Implement expression fiel...
Github user chenghao-intel commented on a diff in the pull request:
https://github.com/apache/spark/pull/16476#discussion_r95281046
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/conditionalExpressions.scala
---
@@ -340,3 +344,102 @@ object CaseKeyWhen {
CaseWhen(cases, elseValue)
}
}
+
+/**
+ * A function that returns the index of expr in (expr1, expr2, ...) list
or 0 if not found.
+ * It takes at least 2 parameters, and all parameters should be subtype of
AtomicType or NullType.
+ * It's also acceptable to give parameters of different types.
+ * If the search string is NULL, the return value is 0 because NULL fails
equality comparison with any value.
+ * When the paramters have different types, comparing will be done based
on type firstly,
+ * for example, ''999'' won't be considered equal with 999, no implicit
cast will be done here.
+ */
+@ExpressionDescription(
+ usage = "_FUNC_(expr, expr1, expr2, ...) - Returns the index of expr in
the expr1, expr2, ... or 0 if not found.",
+ extended = """
+Examples:
+ > SELECT _FUNC_(10, 9, 3, 10, 4);
+ 3
+ > SELECT _FUNC_('a', 'b', 'c', 'd', 'a');
+ 4
+ > SELECT _FUNC_('999', 'a', 999, 9.99, '999');
+ 4
+ """)
+case class Field(children: Seq[Expression]) extends Expression {
+
+ /** Even if expr is not found in (expr1, expr2, ...) list, the value
will be 0, not null */
+ override def nullable: Boolean = false
+ override def foldable: Boolean = children.forall(_.foldable)
+
+ private lazy val ordering =
TypeUtils.getInterpretedOrdering(children(0).dataType)
+
+ private val dataTypeMatchIndex: Seq[Int] = children.tail.zip(Stream from
1).filter(
--- End diff --
Array[Int] instead? Seq[Int] probably a LinkedList in its concrete
implementation.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
