GitHub user kiszk opened a pull request:
https://github.com/apache/spark/pull/21912
[WIP][SPARK-24962][SQL] Refactor CodeGenerator.createUnsafeArray
## What changes were proposed in this pull request?
This PR integrates handling of `UnsafeArrayData` and `GenericArrayData into
one, which is a pair of allocation and assignment. The current
`CodeGenerator.createUnsafeArray` handles only allocation of `UnsafeArrayData`.
This PR introduces a new method `createArrayData` that returns a code to
allocate `UnsafeArrayData` or `GenericArrayData` and to assign a value into the
allocated array.
This PR also reduce the size of generated code by calling a runtime helper.
As a first step, several places where `createUnsafeArray` is used
straightforward are updated by using `createArrayData`.
ToDos
1. Apply `createArrayData` to other places where `createUnsafeArray` is used
1. Merge `createUnsafeArrayWithFallback` into `createArrayData`
1. Use `createArrayData` in `GenArrayData.genCodeToCreateArrayData()`
## How was this patch tested?
Existing tests
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/kiszk/spark SPARK-24962
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/21912.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #21912
----
commit 043aa9d4a587ad904ab2e3af740dd209fbb3ac8a
Author: Kazuaki Ishizaki <ishizaki@...>
Date: 2018-07-29T17:39:40Z
initial commit
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
