[
https://issues.apache.org/jira/browse/SPARK-6368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yin Huai updated SPARK-6368:
----------------------------
Attachment: SchemaBased.nps
Kryo.nps
SchemaBased.nps is the profiling result of the new serializer. You can take a
look at Executor thread.
> Build a specialized serializer for Exchange operator.
> ------------------------------------------------------
>
> Key: SPARK-6368
> URL: https://issues.apache.org/jira/browse/SPARK-6368
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Reporter: Yin Huai
> Assignee: Yin Huai
> Priority: Critical
> Attachments: Kryo.nps, SchemaBased.nps
>
>
> Kryo is still pretty slow because it works on individual objects and relative
> expensive to allocate. For Exchange operator, because the schema for key and
> value are already defined, we can create a specialized serializer to handle
> the specific schemas of key and value.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
