Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/8781#discussion_r39911678
--- Diff:
streaming/src/test/scala/org/apache/spark/streaming/StreamingContextSuite.scala
---
@@ -180,6 +180,38 @@ class StreamingContextSuite extends SparkFunSuite with
BeforeAndAfter with Timeo
assert(ssc.scheduler.isStarted === false)
}
+ test("start should set job group correctly") {
+ ssc = new StreamingContext(conf, batchDuration)
+ ssc.sc.setJobGroup("non-streaming", "non-streaming", true)
+ val sc = ssc.sc
+
+ @volatile var jobGroupFound: String = ""
+ @volatile var jobDescFound: String = ""
+ @volatile var jobInterruptFound: String = ""
+ @volatile var allFound: Boolean = false
+
+ addInputStream(ssc).foreachRDD { rdd =>
+ jobGroupFound = sc.getLocalProperty(SparkContext.SPARK_JOB_GROUP_ID)
+ jobDescFound =
sc.getLocalProperty(SparkContext.SPARK_JOB_DESCRIPTION)
+ jobInterruptFound =
sc.getLocalProperty(SparkContext.SPARK_JOB_INTERRUPT_ON_CANCEL)
+ allFound = true
+ }
+ ssc.start()
+
+ eventually(timeout(5 seconds), interval(10 milliseconds)) {
+ assert(allFound === true)
+ }
--- End diff --
Not necessarily. If it hangs forever Jenkins will time it out anyway. In
general I'm not a huge fan of eventually's because these are usually the ones
we have to end up fixing later; if you guess a timeout wrong then you'll have
to try a slightly higher one after breaking many builds.
In this case it's probably OK to keep this as is, but I'm just saying I
would have written the test differently.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
