Sumeet Sharma created SPARK-34804:
-------------------------------------
Summary: registerFunction shouldnt be logging warning message for
same function being re-registered
Key: SPARK-34804
URL: https://issues.apache.org/jira/browse/SPARK-34804
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 3.1.1
Reporter: Sumeet Sharma
{code:java}
test("function registry warning") {
implicit val ss = spark
import ss.implicits._
val dd = Seq((1),(2)).toDF("a")
val dd1 = udf((i: Int) => i*2)
(1 to 4).foreach {
_ =>
dd.sparkSession.udf.register("function", dd1)
Thread.sleep(1000)
}
dd.withColumn("aa", expr("function(a)")).show(10)
}{code}
logs:
21/03/19 22:39:39 WARN SparkSession$Builder : Using an existing SparkSession;
some spark core configurations may not take effect.
21/03/19 22:39:39 WARN SparkSession$Builder : Using an existing SparkSession;
the static sql configurations will not take effect.
21/03/19 22:39:39 WARN SparkSession$Builder : Using an existing SparkSession;
some spark core configurations may not take effect.
21/03/19 22:39:43 WARN SimpleFunctionRegistry : The function function replaced
a previously registered function.
21/03/19 22:39:44 WARN SimpleFunctionRegistry : The function function replaced
a previously registered function.
21/03/19 22:39:45 WARN SimpleFunctionRegistry : The function function replaced
a previously registered function.
+---+---+
| a| aa|
+---+---+
| 1| 2|
| 2| 4|
+---+---+
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
