Github user mgaido91 commented on a diff in the pull request:
https://github.com/apache/spark/pull/19480#discussion_r145267463
--- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameSuite.scala
---
@@ -2103,4 +2103,16 @@ class DataFrameSuite extends QueryTest with
SharedSQLContext {
testData2.select(lit(7), 'a, 'b).orderBy(lit(1), lit(2), lit(3)),
Seq(Row(7, 1, 1), Row(7, 1, 2), Row(7, 2, 1), Row(7, 2, 2), Row(7,
3, 1), Row(7, 3, 2)))
}
+
+ test("SPARK-22226: splitExpressions should not generate codes beyond
64KB") {
+ val colNumber = 10000
--- End diff --
Yes, this would be the maximum number currently, but #19518 will generate a
great improvement for that. Actually I don't have an exact answer because this
depends on many factors, like which transformations are performed, which are
the datatypes, .... So I am not able to give an answer on that, sorry.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
