Github user bersprockets commented on the issue:
https://github.com/apache/spark/pull/21037
Is array_position a string function? The function array_contains, for
example, works on arrays.
Also, this new function *seems* very similar as the existing instr ("Locate
the position of the first occurrence of substr column in the given string"),
except the return type is BigDecimal rather than Integer.
For example, I ran a query using your branch:
<pre>
scala> val df = Seq((Seq(1, 2, 3), "this is a test"), (Seq(7, 8, 9, 3),
"yeah this")).toDF("a", "b")
df: org.apache.spark.sql.DataFrame = [a: array<int>, b: string]
scala> df.select(array_contains('a, 2), array_position('b, "this"),
instr('b, "this")).show
+--------------------+-----------------------+--------------+
|array_contains(a, 2)|array_position(b, this)|instr(b, this)|
+--------------------+-----------------------+--------------+
| true| 1| 1|
| false| 6| 6|
+--------------------+-----------------------+--------------+
scala>
scala> df.select(array_position('a, 3)).show
<console>:35: error: type mismatch;
found : Int(3)
required: String
df.select(array_position('a, 3)).show
^
scala> df.select(array_contains('a, 3)).show
+--------------------+
|array_contains(a, 3)|
+--------------------+
| true|
| true|
+--------------------+
scala>
</pre>
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
