Guoqiang Li created SPARK-17931: ----------------------------------- Summary: taskScheduler has some unneeded serialization Key: SPARK-17931 URL: https://issues.apache.org/jira/browse/SPARK-17931 Project: Spark Issue Type: Improvement Components: Spark Core Reporter: Guoqiang Li
When taskScheduler instantiates TaskDescription, it calls `Task.serializeWithDependencies(task, sched.sc.addedFiles, sched.sc.addedJars, ser)`. It serializes task and its dependency. But after SPARK-2521 has been merged into the master, the ResultTask class and ShuffleMapTask class no longer contain rdd and closure objects. TaskDescription class can be changed as below: {noformat} class TaskDescription[T]( val taskId: Long, val attemptNumber: Int, val executorId: String, val name: String, val index: Int, val task: Task[T]) extends Serializable {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org