Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/6864#discussion_r32690469
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/sources/hadoopFsRelationSuites.scala
---
@@ -470,6 +470,33 @@ abstract class HadoopFsRelationTest extends QueryTest
with SQLTestUtils {
checkAnswer(sqlContext.table("t"), df.select('b, 'c, 'a).collect())
}
}
+
+ // NOTE: This test suite is not super deterministic. On nodes with only
relatively few cores
+ // (4 or even 1), it's hard to reproduce the data loss issue. But on
nodes with for example 8 or
+ // more cores, the issue can be reproduced steadily. Fortunately our
Jenkins builder meets this
+ // requirement. We probably want to move this test case to
spark-integration-tests or spark-perf
+ // later.
+ test("SPARK-8406") {
--- End diff --
Can you add a description in addition to the JIRA?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
