[
https://issues.apache.org/jira/browse/APEXCORE-478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sandesh updated APEXCORE-478:
-----------------------------
Description:
During deploy time, Apex already knows the type of the object to be serialized
and deserialized, so can optimize the SerDe used. I am quoting Vlad here,
"
+1 for type based serialization. Tuples in most cases are flat
records/pojo and it should be possible programmatically construct a
codec that will significantly outperform Kryo. It should also reduce
amount of data passed over the wire. I started to look in that direction
as well as Kryo serialization is one of bottlenecks that limits Apex
throughput when operators are deployed into different containers
including NODE_LOCAL case."
Link to mail thread on this discussion
http://mail-archives.apache.org/mod_mbox/apex-dev/201605.mbox/%3CCAJhKGzqxknF_7Wj2T2Vfp%3D35cziT0hTJoQTnWopET%2Bj8OgpUJw%40mail.gmail.com%3E
was:
During deploy time, Apex already knows the type of the object to be serialized
and deserialized. I am quoting Vlad here,
"
+1 for type based serialization. Tuples in most cases are flat
records/pojo and it should be possible programmatically construct a
codec that will significantly outperform Kryo. It should also reduce
amount of data passed over the wire. I started to look in that direction
as well as Kryo serialization is one of bottlenecks that limits Apex
throughput when operators are deployed into different containers
including NODE_LOCAL case."
Link to mail thread on this discussion
http://mail-archives.apache.org/mod_mbox/apex-dev/201605.mbox/%3CCAJhKGzqxknF_7Wj2T2Vfp%3D35cziT0hTJoQTnWopET%2Bj8OgpUJw%40mail.gmail.com%3E
> SerDe based on the types
> ------------------------
>
> Key: APEXCORE-478
> URL: https://issues.apache.org/jira/browse/APEXCORE-478
> Project: Apache Apex Core
> Issue Type: Improvement
> Reporter: Sandesh
>
> During deploy time, Apex already knows the type of the object to be
> serialized and deserialized, so can optimize the SerDe used. I am quoting
> Vlad here,
> "
> +1 for type based serialization. Tuples in most cases are flat
> records/pojo and it should be possible programmatically construct a
> codec that will significantly outperform Kryo. It should also reduce
> amount of data passed over the wire. I started to look in that direction
> as well as Kryo serialization is one of bottlenecks that limits Apex
> throughput when operators are deployed into different containers
> including NODE_LOCAL case."
> Link to mail thread on this discussion
> http://mail-archives.apache.org/mod_mbox/apex-dev/201605.mbox/%3CCAJhKGzqxknF_7Wj2T2Vfp%3D35cziT0hTJoQTnWopET%2Bj8OgpUJw%40mail.gmail.com%3E
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
