[
https://issues.apache.org/jira/browse/BEAM-626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15622892#comment-15622892
]
Luke Cwik commented on BEAM-626:
--------------------------------
I'm not for/against fixing AvroCoder to work with Kryo, just pointing out that
the problem is that we lack a spec that says how things need to be serializable
for portability reasons. Until we get the Beam Runner API
[https://issues.apache.org/jira/browse/BEAM-115] up and going we will continue
to run into these issues.
Dataflow relied on Java serialization for DoFns, and Jackson for Coders. Spark
relies on Kryo. Another runner may pull in yet another way as to how they
serialize DoFns/Coders/etc...
Still looking at the PR.
> AvroCoder not deserializing correctly in Kryo
> ---------------------------------------------
>
> Key: BEAM-626
> URL: https://issues.apache.org/jira/browse/BEAM-626
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
> Reporter: Aviem Zur
> Assignee: Aviem Zur
> Priority: Minor
>
> Unlike with Java serialization, when deserializing AvroCoder using Kryo, the
> resulting AvroCoder is missing all of its transient fields.
> The reason it works with Java serialization is because of the usage of
> writeReplace and readResolve, which Kryo does not adhere to.
> In ProtoCoder for example there are also unserializable members, the way it
> is solved there is lazy initializing these members via their getters, so they
> are initialized in the deserialized object on first call to the member.
> It seems AvroCoder is the only class in Beam to use writeReplace convention.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
