[
https://issues.apache.org/jira/browse/SPARK-12778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15094090#comment-15094090
]
Ted Yu commented on SPARK-12778:
--------------------------------
I have seen SPARK-12555 but it didn't identify where in the codebase the
problem is.
I checked unsafe/src/main/java/org/apache/spark/unsafe/Platform.java in master
branch where endianness is not taken care of.
> Use of Java Unsafe should take endianness into account
> ------------------------------------------------------
>
> Key: SPARK-12778
> URL: https://issues.apache.org/jira/browse/SPARK-12778
> Project: Spark
> Issue Type: Bug
> Components: Input/Output
> Reporter: Ted Yu
>
> In Platform.java, methods of Java Unsafe are called directly without
> considering endianness.
> In thread, 'Tungsten in a mixed endian environment', Adam Roberts reported
> data corruption when "spark.sql.tungsten.enabled" is enabled in mixed endian
> environment.
> Platform.java should take endianness into account.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
