Guoqiang Li created SPARK-17931:

             Summary: taskScheduler has some unneeded serialization
                 Key: SPARK-17931
             Project: Spark
          Issue Type: Improvement
          Components: Spark Core
            Reporter: Guoqiang Li

When taskScheduler instantiates TaskDescription, it calls 
ser)`.  It serializes task and its dependency. 

But after SPARK-2521 has been merged into the master, the ResultTask class and 
ShuffleMapTask  class no longer contain rdd and closure objects. 
TaskDescription class can be changed as below:

class TaskDescription[T](
val taskId: Long,
val attemptNumber: Int,
val executorId: String,
val name: String,
val index: Int, 
val task: Task[T]) extends Serializable

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

Reply via email to