mgaido91 commented on a change in pull request #25642: [SPARK-28916][SQL] Split
subexpression elimination functions code
URL: https://github.com/apache/spark/pull/25642#discussion_r320766329
##########
File path: sql/core/src/test/scala/org/apache/spark/sql/DataFrameSuite.scala
##########
@@ -2202,4 +2202,12 @@ class DataFrameSuite extends QueryTest with
SharedSparkSession {
|*(1) Range (0, 10, step=1, splits=2)""".stripMargin))
}
}
+
+ test("SPARK-28916: subexrepssion elimination can cause 64kb code limit") {
+ val df = spark.range(2).selectExpr((0 to 5000).map(i => s"id as
field_$i"): _*)
+ df.createOrReplaceTempView("spark64kb")
+ val data = spark.sql("select * from spark64kb limit 10")
+ // This fails if 64Kb limit is reached in code generation
+ data.describe()
Review comment:
I need to revisit it according to Wenchen's suggestion... unfortunately this
might take some time as I am a bit busy these days...
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
