tanelk commented on a change in pull request #29577:
URL: https://github.com/apache/spark/pull/29577#discussion_r479794502
##########
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
##########
@@ -541,7 +541,8 @@ object Overlay {
Spark ANSI SQL
> SELECT _FUNC_(encode('Spark SQL', 'utf-8') PLACING encode('tructured',
'utf-8') FROM 2 FOR 4);
Structured SQL
- """)
+ """,
+ since = "3.0.0")
Review comment:
In master there are currently 67 expressions without the since tag:
```
test("Since has a valid value") {
val badExpressions = spark.sessionState.functionRegistry.listFunction()
.map(spark.sessionState.catalog.lookupFunctionInfo)
.filter(funcInfo =>
!funcInfo.getSince.matches("[0-9]+\\.[0-9]+\\.[0-9]+"))
.map(_.getClassName)
.distinct
.sorted
if (badExpressions.nonEmpty) {
fail(s"${badExpressions.length} expressions with invalid 'since':\n"
+ badExpressions.mkString("\n"))
}
}
```
```
[info] - Since has a valid value *** FAILED *** (16 milliseconds)
[info] 67 expressions with invalid 'since':
[info] org.apache.spark.sql.catalyst.expressions.Abs
[info] org.apache.spark.sql.catalyst.expressions.Add
[info] org.apache.spark.sql.catalyst.expressions.And
[info] org.apache.spark.sql.catalyst.expressions.ArrayContains
[info] org.apache.spark.sql.catalyst.expressions.AssertTrue
[info] org.apache.spark.sql.catalyst.expressions.BitwiseAnd
[info] org.apache.spark.sql.catalyst.expressions.BitwiseNot
[info] org.apache.spark.sql.catalyst.expressions.BitwiseOr
[info] org.apache.spark.sql.catalyst.expressions.BitwiseXor
[info] org.apache.spark.sql.catalyst.expressions.CallMethodViaReflection
[info] org.apache.spark.sql.catalyst.expressions.CaseWhen
[info] org.apache.spark.sql.catalyst.expressions.Cast
[info] org.apache.spark.sql.catalyst.expressions.Concat
[info] org.apache.spark.sql.catalyst.expressions.Crc32
[info] org.apache.spark.sql.catalyst.expressions.CreateArray
[info] org.apache.spark.sql.catalyst.expressions.CreateMap
[info] org.apache.spark.sql.catalyst.expressions.CreateNamedStruct
[info] org.apache.spark.sql.catalyst.expressions.CurrentDatabase
[info] org.apache.spark.sql.catalyst.expressions.Divide
[info] org.apache.spark.sql.catalyst.expressions.EqualNullSafe
[info] org.apache.spark.sql.catalyst.expressions.EqualTo
[info] org.apache.spark.sql.catalyst.expressions.Explode
[info] org.apache.spark.sql.catalyst.expressions.GetJsonObject
[info] org.apache.spark.sql.catalyst.expressions.GreaterThan
[info] org.apache.spark.sql.catalyst.expressions.GreaterThanOrEqual
[info] org.apache.spark.sql.catalyst.expressions.Greatest
[info] org.apache.spark.sql.catalyst.expressions.If
[info] org.apache.spark.sql.catalyst.expressions.In
[info] org.apache.spark.sql.catalyst.expressions.Inline
[info] org.apache.spark.sql.catalyst.expressions.InputFileBlockLength
[info] org.apache.spark.sql.catalyst.expressions.InputFileBlockStart
[info] org.apache.spark.sql.catalyst.expressions.InputFileName
[info] org.apache.spark.sql.catalyst.expressions.JsonTuple
[info] org.apache.spark.sql.catalyst.expressions.Least
[info] org.apache.spark.sql.catalyst.expressions.LessThan
[info] org.apache.spark.sql.catalyst.expressions.LessThanOrEqual
[info] org.apache.spark.sql.catalyst.expressions.MapKeys
[info] org.apache.spark.sql.catalyst.expressions.MapValues
[info] org.apache.spark.sql.catalyst.expressions.Md5
[info] org.apache.spark.sql.catalyst.expressions.MonotonicallyIncreasingID
[info] org.apache.spark.sql.catalyst.expressions.Multiply
[info] org.apache.spark.sql.catalyst.expressions.Murmur3Hash
[info] org.apache.spark.sql.catalyst.expressions.Not
[info] org.apache.spark.sql.catalyst.expressions.Or
[info] org.apache.spark.sql.catalyst.expressions.Overlay
[info] org.apache.spark.sql.catalyst.expressions.Pmod
[info] org.apache.spark.sql.catalyst.expressions.PosExplode
[info] org.apache.spark.sql.catalyst.expressions.Remainder
[info] org.apache.spark.sql.catalyst.expressions.Sha1
[info] org.apache.spark.sql.catalyst.expressions.Sha2
[info] org.apache.spark.sql.catalyst.expressions.Size
[info] org.apache.spark.sql.catalyst.expressions.SortArray
[info] org.apache.spark.sql.catalyst.expressions.SparkPartitionID
[info] org.apache.spark.sql.catalyst.expressions.Stack
[info] org.apache.spark.sql.catalyst.expressions.Subtract
[info] org.apache.spark.sql.catalyst.expressions.TimeWindow
[info] org.apache.spark.sql.catalyst.expressions.UnaryMinus
[info] org.apache.spark.sql.catalyst.expressions.UnaryPositive
[info] org.apache.spark.sql.catalyst.expressions.Uuid
[info] org.apache.spark.sql.catalyst.expressions.xml.XPathBoolean
[info] org.apache.spark.sql.catalyst.expressions.xml.XPathDouble
[info] org.apache.spark.sql.catalyst.expressions.xml.XPathFloat
[info] org.apache.spark.sql.catalyst.expressions.xml.XPathInt
[info] org.apache.spark.sql.catalyst.expressions.xml.XPathList
[info] org.apache.spark.sql.catalyst.expressions.xml.XPathLong
[info] org.apache.spark.sql.catalyst.expressions.xml.XPathShort
[info] org.apache.spark.sql.catalyst.expressions.xml.XPathString
(ExpressionInfoSuite.scala:204)
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
