Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/7674#discussion_r35500606
--- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameSuite.scala
---
@@ -825,11 +813,43 @@ class DataFrameSuite extends QueryTest with
SQLTestUtils {
assert(e2.getMessage.contains("Inserting into an RDD-based table is
not allowed."))
// error case: insert into an OneRowRelation
- new DataFrame(ctx, OneRowRelation).registerTempTable("one_row")
+ new DataFrame(sqlContext,
OneRowRelation).registerTempTable("one_row")
val e3 = intercept[AnalysisException] {
insertion.write.insertInto("one_row")
}
assert(e3.getMessage.contains("Inserting into an RDD-based table is
not allowed."))
}
}
+
+ test("SPARK-8608: call `show` on local DataFrame with random columns
should return same value") {
+ // Make sure we can pass this test for both codegen mode and
interpreted mode.
+ withSQLConf(SQLConf.CODEGEN_ENABLED.key -> "true") {
+ val df = testData.select(rand(33))
+ assert(df.showString(5) == df.showString(5))
+ }
+
+ withSQLConf(SQLConf.CODEGEN_ENABLED.key -> "false") {
+ val df = testData.select(rand(33))
+ assert(df.showString(5) == df.showString(5))
+ }
+
+ // We will reuse the same Expression object for LocalRelation.
+ val df = (1 to 10).map(Tuple1.apply).toDF().select(rand(33))
+ assert(df.showString(5) == df.showString(5))
+ }
+
+ test("SPARK-8609: local DataFrame with random columns should return same
value after sort") {
--- End diff --
to make it more clear, now this PR is aimed to bug fix, i.e. fix
nondeterministic expression handling for local projection and sort. Leave #7593
to optimization, i.e. remove still-need-evaluate expressions from `Sort`.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
