MaxGekk commented on code in PR #39719:
URL: https://github.com/apache/spark/pull/39719#discussion_r1174354578
##########
sql/core/src/test/scala/org/apache/spark/sql/CsvFunctionsSuite.scala:
##########
@@ -577,4 +578,11 @@ class CsvFunctionsSuite extends QueryTest with
SharedSparkSession {
$"csv", schema_of_csv("1,2\n2"), Map.empty[String, String].asJava))
checkAnswer(actual, Row(Row(1, "2\n2")))
}
+
+ test("StructsToCsv should not generate codes beyond 64KB") {
Review Comment:
Could you clarify this test title, please. I don't see anything related to
64KB in the test.
Doesn't the test just duplicates existing one: `to_csv - struct`?
##########
sql/core/src/test/scala/org/apache/spark/sql/CsvFunctionsSuite.scala:
##########
@@ -577,4 +578,11 @@ class CsvFunctionsSuite extends QueryTest with
SharedSparkSession {
$"csv", schema_of_csv("1,2\n2"), Map.empty[String, String].asJava))
checkAnswer(actual, Row(Row(1, "2\n2")))
}
+
+ test("StructsToCsv should not generate codes beyond 64KB") {
+ val range = Range.inclusive(1, 5000)
+ val struct = CreateStruct.create(range.map(Literal.apply))
+ val expected = range.mkString(",")
+ checkEvaluation(StructsToCsv(Map.empty, struct), expected)
Review Comment:
It is better to place the test to `CsvExpressionsSuite` because
`CsvFunctionsSuite` aims to check functions like `to_csv`/`from_csv`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
