[
https://issues.apache.org/jira/browse/DRILL-5846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16419544#comment-16419544
]
ASF GitHub Bot commented on DRILL-5846:
---------------------------------------
Github user sachouche commented on the issue:
https://github.com/apache/drill/pull/1060
I have updated this pull request with the following changes:
- Excluded the implicit column optimizations from this pull request (will
be included as part of another Drill Jira)
- Tuned a threshold (16bytes vs 1024) on when to use "putByte() vs
copyMemory()" based on @vrozov findings when benchmarking using JMH
- There were lingering questions on the use of byte arrays for intermediary
processing; after a discussion with @vrozov we agreed to tackle such an
analysis within a new JIRA
([DRILL-6301](https://issues.apache.org/jira/browse/DRILL-6301?filter=-1)) so
that the Drill customer can immediately rip the benefits of the current
enhancements
- Moved the newly added MemoryUtils class to the Parquet flat reader
project (made it package private); this is temporary till DRILL-6301 is
resolved.
> Improve Parquet Reader Performance for Flat Data types
> -------------------------------------------------------
>
> Key: DRILL-5846
> URL: https://issues.apache.org/jira/browse/DRILL-5846
> Project: Apache Drill
> Issue Type: Improvement
> Components: Storage - Parquet
> Affects Versions: 1.11.0
> Reporter: salim achouche
> Assignee: salim achouche
> Priority: Major
> Labels: performance
> Fix For: 1.14.0
>
>
> The Parquet Reader is a key use-case for Drill. This JIRA is an attempt to
> further improve the Parquet Reader performance as several users reported that
> Parquet parsing represents the lion share of the overall query execution. It
> tracks Flat Data types only as Nested DTs might involve functional and
> processing enhancements (e.g., a nested column can be seen as a Document;
> user might want to perform operations scoped at the document level that is no
> need to span all rows). Another JIRA will be created to handle the nested
> columns use-case.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
