GitHub user viirya opened a pull request:
https://github.com/apache/spark/pull/20984
[SPARK-23875][SQL] Add IndexedSeq wrapper for ArrayData
## What changes were proposed in this pull request?
We don't have a good way to sequentially access `UnsafeArrayData` with a
common interface such as `Seq`. An example is `MapObject` where we need to
access several sequence collection types together. But `UnsafeArrayData`
doesn't implement `ArrayData.array`. Calling `toArray` will copy the entire
array. We can provide an `IndexedSeq` wrapper for `ArrayData`, so we can avoid
copying the entire array.
## How was this patch tested?
TBD.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/viirya/spark-1 SPARK-23875
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/20984.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #20984
----
commit 27eddd764bc60f9f41156c3548e98e6aea68fd02
Author: Liang-Chi Hsieh <viirya@...>
Date: 2018-04-05T14:33:39Z
Add IndexedSeq wrapper for ArrayData.
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
