[
https://issues.apache.org/jira/browse/DRILL-5846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16441854#comment-16441854
]
ASF GitHub Bot commented on DRILL-5846:
---------------------------------------
Github user sachouche commented on the issue:
https://github.com/apache/drill/pull/1060
@parthchandra , @vrozov
I have done the following modifications:
- Renamed newly added files with the prefix "VL" with "VarLen" as suggested
by @parthchandra
- After talking offline with @vrozov about his objections to the
MemoryUtils and also from previous feedbacks from @parthchandra , I have a)
deleted this utility, b) exposed the needed functionality under the class
DrillBuf (using Netty API's), and c) used the same configuration to control the
checks.
@parthchandra , @vrozov, please review this
[document](https://docs.google.com/document/d/1BSNem_ItP-Vxlr6auSP_iwwOLM9rwWZYxGwCsXi-IE8/edit?usp=sharing)
and the associated JMH code [here](https://github.com/sachouche/drill-jmh).
This should provide you with performance benchmarks regarding the bulk approach
that I have used in this PR request (please focus on test-3 which is Parquet
specific); I will be adding more fine-grained tests with regard to the Memory
Access tests (requested by @vrozov).
> Improve Parquet Reader Performance for Flat Data types
> -------------------------------------------------------
>
> Key: DRILL-5846
> URL: https://issues.apache.org/jira/browse/DRILL-5846
> Project: Apache Drill
> Issue Type: Improvement
> Components: Storage - Parquet
> Affects Versions: 1.11.0
> Reporter: salim achouche
> Assignee: salim achouche
> Priority: Major
> Labels: performance
> Fix For: 1.14.0
>
> Attachments: 2542d447-9837-3924-dd12-f759108461e5.sys.drill,
> 2542d49b-88ef-38e3-a02b-b441c1295817.sys.drill
>
>
> The Parquet Reader is a key use-case for Drill. This JIRA is an attempt to
> further improve the Parquet Reader performance as several users reported that
> Parquet parsing represents the lion share of the overall query execution. It
> tracks Flat Data types only as Nested DTs might involve functional and
> processing enhancements (e.g., a nested column can be seen as a Document;
> user might want to perform operations scoped at the document level that is no
> need to span all rows). Another JIRA will be created to handle the nested
> columns use-case.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
