Guoqiang Li created SPARK-17930:

             Summary: The SerializerInstance instance used when deserializing a 
TaskResult is not reused 
                 Key: SPARK-17930
             Project: Spark
          Issue Type: Improvement
          Components: Spark Core
    Affects Versions: 2.0.1, 1.6.1
            Reporter: Guoqiang Li

The following code is called when the DirectTaskResult instance is deserialized

  def value(): T = {
    if (valueObjectDeserialized) {
    } else {
      // Each deserialization creates a new instance of SerializerInstance, 
which is very time-consuming
      val resultSer = SparkEnv.get.serializer.newInstance()
      valueObject = resultSer.deserialize(valueBytes)
      valueObjectDeserialized = true

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

Reply via email to