NarekDW commented on code in PR #39719:
URL: https://github.com/apache/spark/pull/39719#discussion_r1174382698
##########
sql/core/src/test/scala/org/apache/spark/sql/CsvFunctionsSuite.scala:
##########
@@ -577,4 +578,11 @@ class CsvFunctionsSuite extends QueryTest with
SharedSparkSession {
$"csv", schema_of_csv("1,2\n2"), Map.empty[String, String].asJava))
checkAnswer(actual, Row(Row(1, "2\n2")))
}
+
+ test("StructsToCsv should not generate codes beyond 64KB") {
Review Comment:
Sure, `checkEvaluation` will execute this test in both modes without codegen
and with codegen modes. In case of codegen mode, if the generated java code
(method) will be larger than 64 kb in size it won't be able to compile it, and
the test will fail, as Java has a 64kb limit on the size of methods.
It doesn't duplicate `to_csv - struct` test case, because the purpose of
this test - is to generate big StructType(with 5000 literals in current case)
and test it on `StructsToCsv` expression to be sure that generated java code
doesn't exceed the limit of 64 kb in size.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
