[
https://issues.apache.org/jira/browse/SPARK-4802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14257668#comment-14257668
]
Tathagata Das commented on SPARK-4802:
--------------------------------------
SPARK-2892 is not a duplicate of this though they have same symptoms, the
reason are different but related. SPARK-2892 affects only socket receiver and
is probably because the socket receiver does not stop cleanly. SPARK-4802
affects all receivers, and prevents the driver from realizing that all
receivers have been closed.
> ReceiverInfo removal at ReceiverTracker upon deregistering receiver
> -------------------------------------------------------------------
>
> Key: SPARK-4802
> URL: https://issues.apache.org/jira/browse/SPARK-4802
> Project: Spark
> Issue Type: Bug
> Components: Streaming
> Affects Versions: 1.1.0
> Reporter: Ilayaperumal Gopinathan
> Priority: Minor
>
> When the streaming receiver is deregistered, the ReceiverTracker doesn't
> remove the corresponding receiverInfo entry for the receiver.
> When the receiver is stopped at the executor and the ReceiverTrackerActor
> that processes the 'DeregisterReceiver' message. Shouldn't it remove the
> receiverInfo entry for that receiver as the receiver is actually deregistered?
> Not sure if there is any specific reason for not removing it.
> Currently, I see this warning if we don't remove it:
> WARN main-EventThread scheduler.ReceiverTracker - All of the receivers have
> not deregistered, Map(0 ->
> ReceiverInfo(0,MyReceiver-0,null,false,localhost,Stopped by driver,))
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
