GitHub user sameeragarwal opened a pull request:
https://github.com/apache/spark/pull/13832
[SPARK-16123] Avoid NegativeArraySizeException while reserving additional
capacity in VectorizedColumnReader
## What changes were proposed in this pull request?
This patch fixes an overflow bug in vectorized parquet reader where both
off-heap and on-heap variants of `ColumnVector.reserve()` can unfortunately
overflow while reserving additional capacity during reads.
## How was this patch tested?
Manual Tests
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/sameeragarwal/spark negative-array
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/13832.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #13832
----
commit a5ec358274f270c6edad65dc89de4181fd7d2dc9
Author: Sameer Agarwal <[email protected]>
Date: 2016-06-22T00:59:27Z
Avoid NegativeArraySizeException while reserving additional capacity in
VectorizedColumnReader
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
