Github user dmcguire81 commented on a diff in the pull request:
https://github.com/apache/spark/pull/5559#discussion_r28641760
--- Diff:
streaming/src/test/scala/org/apache/spark/streaming/ReceiverSuite.scala ---
@@ -176,7 +176,7 @@ class ReceiverSuite extends TestSuiteBase with Timeouts
with Serializable {
blockGenerator.addData(count)
generatedData += count
count += 1
- Thread.sleep(1)
+ Thread.`yield`()
--- End diff --
I've tried all number of micro-waits, and nothing seems to do the trick
like Thread.sleep(0). My hunch is that, given a longer test, the preemptive
multithreading would even out and each thread would see sufficient CPU time,
but, for such a concise run, the main test thread needs to be voluntarily
cooperative.
At any rate, there doesn't seem to be a good compromise between platform
independence, intuitive appeal, and speed and repeatability of the tests, so
I'm at a bit of a loss. Currently this defect is hamstringing our entire
stream-processing operation (rate limiting doesn't work reliably at any rate,
but just crashes obviously and violently for the documented rates), so I'd
appreciate any guidance or shepherding you can provide.
Cheers!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
