GitHub user rdblue opened a pull request:

    https://github.com/apache/spark/pull/21070

    SPARK-23972: Update Parquet to 1.10.0.

    ## What changes were proposed in this pull request?
    
    This updates Parquet to 1.10.0 and updates the vectorized path for buffer 
management changes. Parquet 1.10.0 uses ByteBufferInputStream instead of byte 
arrays in encoders. This allows Parquet to break allocations into smaller 
chunks that are better for garbage collection.
    
    ## How was this patch tested?
    
    Existing Parquet tests.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/rdblue/spark 
SPARK-23972-update-parquet-to-1.10.0

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/spark/pull/21070.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #21070
    
----
commit 4df17a6e9726cb22e499d479a9ab48f5db18a538
Author: Ryan Blue <blue@...>
Date:   2017-12-01T01:25:53Z

    SPARK-23972: Update Parquet to 1.10.0.
    
    This updates the vectorized path for changes in Parquet 1.10.0, which
    uses ByteBufferInputStream instead of byte arrays in encoders. This
    allows Parquet to break allocations into smaller chunks that are better
    for garbage collection.

----


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

Reply via email to