GitHub user viirya opened a pull request:
https://github.com/apache/spark/pull/21312
[SPARK-24259][SQL] ArrayWriter for Arrow produces wrong output
## What changes were proposed in this pull request?
Right now `ArrayWriter` used to output Arrow data for array type, doesn't
do `clear` or `reset` after each batch. It produces wrong output.
## How was this patch tested?
Added test.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/viirya/spark-1 SPARK-24259
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/21312.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #21312
----
commit 093728ef75f4cecbac5d5f4f82fcce0cc47759b5
Author: Liang-Chi Hsieh <viirya@...>
Date: 2018-05-13T00:29:09Z
Call clear after each batch.
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
