[
https://issues.apache.org/jira/browse/SPARK-12785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15101520#comment-15101520
]
Tim Preece commented on SPARK-12785:
------------------------------------
I notice there is an explicit check for endianess in OffHeapColumnVector.
Is the a technical reason why this feature is only being developed for Little
Endian platforms ?
I think the endianess should never matter ( providing Spark is run on an
homogeneous endianess cluster ).
> Implement columnar in memory representation
> -------------------------------------------
>
> Key: SPARK-12785
> URL: https://issues.apache.org/jira/browse/SPARK-12785
> Project: Spark
> Issue Type: New Feature
> Components: SQL
> Reporter: Nong Li
> Assignee: Nong Li
> Fix For: 2.0.0
>
>
> Tungsten can benefit from having a columnar in memory representation which
> can provide a few benefits:
> - Enables vectorized execution
> - Improves memory efficiency (memory is more tightly packed)
> - Enables cheap serialization/zero-copy transfer with third party components
> (e.g. numpy)
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
