[
https://issues.apache.org/jira/browse/SPARK-3283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tathagata Das resolved SPARK-3283.
----------------------------------
Resolution: Fixed
Fix Version/s: 1.5.0
> Receivers sometimes do not get spread out to multiple nodes
> -----------------------------------------------------------
>
> Key: SPARK-3283
> URL: https://issues.apache.org/jira/browse/SPARK-3283
> Project: Spark
> Issue Type: Bug
> Components: Streaming
> Reporter: Tathagata Das
> Assignee: Tathagata Das
> Fix For: 1.5.0
>
>
> The probable reason this happens is because the JobGenerator and JobScheduler
> start generating jobs with tasks. When the ReceiverTracker submits the task
> containing receivers, the tasks get assigned according to empty slots, which
> may be instantaneously available on one node, instead of all the nodes.
> The original behavior was that the jobs started only after the receivers are
> started, thus ensuring that all the slots are free and the receivers are
> spread evenly across all the nodes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
