Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/7600#discussion_r35699524
--- Diff:
streaming/src/test/scala/org/apache/spark/streaming/scheduler/ReceiverTrackerSuite.scala
---
@@ -81,17 +89,25 @@ private class RateLimitInputDStream(@transient ssc_ :
StreamingContext)
}
/**
- * A Receiver as an object so we can read its rate limit.
+ * A Receiver as an object so we can read its rate limit. Make sure to
call `reset()` when
+ * reusing this receiver, otherwise a non-null `executor_` field will
prevent it from being
+ * serialized when receivers are installed on executors.
*
* @note It's necessary to be a top-level object, or else serialization
would create another
* one on the executor side and we won't be able to read its rate
limit.
*/
-private object SingletonDummyReceiver extends DummyReceiver(0)
+private[streaming] object SingletonDummyReceiver extends DummyReceiver(0) {
+
+ /** Reset the object to be usable in another test. */
+ def reset(): Unit = {
+ executor_ = null
+ }
+}
/**
* Dummy receiver implementation
*/
-private class DummyReceiver(receiverId: Int, host: Option[String] = None)
+private[streaming] class DummyReceiver(receiverId: Int, host:
Option[String] = None)
--- End diff --
Since this receiver and associated InputDStream is used in multiple
testsuites, mind renaming this to a more contextually relevant name than
`Dummy`? How about `RateTestReceiver` and `RateTestInputDStream`?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
