Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/20963#discussion_r179934861
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonSuite.scala
---
@@ -2127,4 +2127,39 @@ class JsonSuite extends QueryTest with
SharedSQLContext with TestJsonData {
assert(df.schema === expectedSchema)
}
}
+
+ test("SPARK-23849: schema inferring touches less data if samplingRation
< 1.0") {
+ val predefinedSample = Set[Int](2, 8, 15, 27, 30, 34, 35, 37, 44, 46,
+ 57, 62, 68, 72)
--- End diff --
Not need to have so many elements in this set. Please combine the tests in
your CSV PR.
Instead of calling `json()`, we can do it using `format("json")`. Then, you
can combine the test cases for both CSV and Json.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
