Github user viirya commented on the pull request:
https://github.com/apache/spark/pull/4467#issuecomment-90414046
Currently, `Receiver.onStart()` is called before it sends
`RegisterReceiver` message to `ReceiverSupervisor` for registration.
In `Receiver.onStart()`, it already begins to pull data. So in your cases,
receiver {C} is possibly to lose its data.
We know that the thing was getting stuck because sometimes receiver will
register after tracker is going to stop. So the receiver will not get stop
message properly.
As you said, we can disallow its registration. However, because it already
pulls data, it may lose received data.
This solution I proposed, also disallows receiver to register. Besides, it
moves registration ahead of starting. So a receiver is only going to pull data
after it is registered with the tracker. By doing that, we guarantee no data
loss for receivers.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
