Eren Avsarogullari created SPARK-17894:

             Summary: Uniqueness of TaskSetManager name
                 Key: SPARK-17894
             Project: Spark
          Issue Type: Bug
          Components: Scheduler
    Affects Versions: 2.1.0
            Reporter: Eren Avsarogullari

TaskSetManager should have unique name to avoid adding duplicate ones to *Pool* 
via *SchedulableBuilder*. This problem surfaced with and please find discussion:

There is 1x1 relationship between Stage Attempt Id and TaskSetManager so 
taskSet.Id covering both stageId and stageAttemptId looks to be used for 
TaskSetManager as well. 

What do you think about proposed TaskSetManager Name?

*Current TaskSetManager Name* : 
{code:java} var name = "TaskSet_" + taskSet.stageId.toString{code}
*Sample*: TaskSet_0

*Proposed TaskSetManager Name* : 
{code:java} var name = "TaskSet_" + taskSet.Id (stageId + "." + stageAttemptId) 
*Sample* : TaskSet_0.0

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

Reply via email to