Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/3026#discussion_r19694569
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/scheduler/ReceiverTracker.scala
---
@@ -48,23 +49,28 @@ private[streaming] case class
DeregisterReceiver(streamId: Int, msg: String, err
* This class manages the execution of the receivers of
NetworkInputDStreams. Instance of
* this class must be created after all input streams have been added and
StreamingContext.start()
* has been called because it needs the final set of input streams at the
time of instantiation.
+ *
+ * @param skipReceiverLaunch Do not launch the receiver. This is useful
for testing.
*/
private[streaming]
-class ReceiverTracker(ssc: StreamingContext) extends Logging {
+class ReceiverTracker(ssc: StreamingContext, skipReceiverLaunch: Boolean =
false) extends Logging {
- val receiverInputStreams = ssc.graph.getReceiverInputStreams()
- val receiverInputStreamMap = Map(receiverInputStreams.map(x => (x.id,
x)): _*)
- val receiverExecutor = new ReceiverLauncher()
- val receiverInfo = new HashMap[Int, ReceiverInfo] with
SynchronizedMap[Int, ReceiverInfo]
- val receivedBlockInfo = new HashMap[Int,
SynchronizedQueue[ReceivedBlockInfo]]
- with SynchronizedMap[Int, SynchronizedQueue[ReceivedBlockInfo]]
- val timeout = AkkaUtils.askTimeout(ssc.conf)
- val listenerBus = ssc.scheduler.listenerBus
+ private val receiverInputStreams = ssc.graph.getReceiverInputStreams()
+ private val receiverInputStreamMap = Map(receiverInputStreams.map(x =>
(x.id, x)): _*)
--- End diff --
Just removed this. Realized that it is not needed. Replaced it with just a
sequence of stream ids.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
