[
https://issues.apache.org/jira/browse/SPARK-27416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16886535#comment-16886535
]
Josh Rosen edited comment on SPARK-27416 at 7/16/19 10:56 PM:
--------------------------------------------------------------
I think that we should backport this for Spark 2.4.4 because the related
SPARK-27406 was backported for Spark 2.4.3 and both bugs seem like potential
correctness issues (they'll often result in crashes but might theoretically
result in wrong results in some cases, something that we saw happen with the
original UnsafeRow OOPs bug).
I spotted this while going through the full list of 2.4.0 -> 2.4.3 commits: I
saw that the related ArrayData PR was backported but this followup for Maps was
not.
I can open a backport PR later; if anyone else wants to pick this up then
please feel free to.
/cc [~smilegator]
was (Author: joshrosen):
I think that we should backport this for Spark 2.4.4 because the related
SPARK-27406 was backported for Spark 2.4.3 and both bugs seem like potential
correctness issues (they'll often result in crashes but might theoretically
result in wrong results in some cases, something that we saw happen with the
original UnsafeRow OOPs bug).
I spotted this while going through the full list of 2.4.0 -> 2.4.3 commits.
I can open a backport PR later; if anyone else wants to pick this up then
please feel free to.
/cc [~smilegator]
> UnsafeMapData & UnsafeArrayData Kryo serialization breaks when two machines
> have different Oops size
> ----------------------------------------------------------------------------------------------------
>
> Key: SPARK-27416
> URL: https://issues.apache.org/jira/browse/SPARK-27416
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 2.4.1
> Reporter: peng bo
> Assignee: peng bo
> Priority: Major
> Fix For: 3.0.0
>
>
> Actually this's follow up for
> https://issues.apache.org/jira/browse/SPARK-27406,
> https://issues.apache.org/jira/browse/SPARK-10914
> This issue is to fix the UnsafeMapData & UnsafeArrayData Kryo serialization
> issue when two machines have different Oops size.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
