[
https://issues.apache.org/jira/browse/SPARK-4802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ilayaperumal Gopinathan updated SPARK-4802:
-------------------------------------------
Description:
When the streaming receiver is deregistered, the ReceiverTracker doesn't remove
the corresponding receiverInfo entry for the receiver.
When the receiver is stopped at the executor and the ReceiverTrackerActor that
processes the 'DeregisterReceiver' message. Shouldn't it remove the
receiverInfo entry for that receiver as the receiver is actually deregistered?
Not sure if there is any specific reason for not removing it.
Currently, I see this warning if we don't remove it:
WARN main-EventThread scheduler.ReceiverTracker - All of the receivers have not
deregistered, Map(0 -> ReceiverInfo(0,MyReceiver-0,null,false,localhost,Stopped
by driver,))
was:
When the streaming receiver is deregistered, the ReceiverTracker doesn't remove
the corresponding receiverInfo entry for the receiver.
When the receiver is stopped at the executor and the ReceiverTrackerActor that
processes the 'DeregisterReceiver' message. Shouldn't it remove the
receiverInfo entry for that receiver as the receiver is actually deregistered?
Not sure if there is any specific reason for not removing it.
> ReceiverInfo removal at ReceiverTracker upon deregistering receiver
> -------------------------------------------------------------------
>
> Key: SPARK-4802
> URL: https://issues.apache.org/jira/browse/SPARK-4802
> Project: Spark
> Issue Type: Bug
> Components: Streaming
> Affects Versions: 1.1.0
> Reporter: Ilayaperumal Gopinathan
> Priority: Minor
>
> When the streaming receiver is deregistered, the ReceiverTracker doesn't
> remove the corresponding receiverInfo entry for the receiver.
> When the receiver is stopped at the executor and the ReceiverTrackerActor
> that processes the 'DeregisterReceiver' message. Shouldn't it remove the
> receiverInfo entry for that receiver as the receiver is actually deregistered?
> Not sure if there is any specific reason for not removing it.
> Currently, I see this warning if we don't remove it:
> WARN main-EventThread scheduler.ReceiverTracker - All of the receivers have
> not deregistered, Map(0 ->
> ReceiverInfo(0,MyReceiver-0,null,false,localhost,Stopped by driver,))
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
