Github user sachouche commented on the issue:
https://github.com/apache/drill/pull/1060
Parth,
- I have attached, within the DRILL-5846, two profiles with latest Apache
code and this PR request (bounds checks are off):
o Used one thread in each run
o I observe ~3x performance difference when the new logic is turned on
o The difference is 4x if I include the implicit column optimization
(which is not part of this PR)
o The impact of the new optimizations can be felt when there are many
variable length columns
- The rational of trying to approve this PR
o The optimizations that I have included are local to the Flat Parquet
Reader (incapsulated)
o The logic is backward compatible and turned off by default
o I have added the new Batch Sizing functionality on top of this PR
(columnar processing pattern)
o The result of DRILL-6301 would only result in a local refactoring step
o Not being able to add the new code results in a substantial
maintenance overhead
---
