[GitHub] drill pull request #1060: DRILL-5846: Improve parquet performance for Flat D...

2018-04-04 Thread vrozov
Github user vrozov commented on a diff in the pull request: https://github.com/apache/drill/pull/1060#discussion_r179344792 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/MemoryUtils.java --- @@ -0,0 +1,199 @@ +/** --- End diff --

[GitHub] drill pull request #1060: DRILL-5846: Improve parquet performance for Flat D...

2018-04-04 Thread vrozov
Github user vrozov commented on a diff in the pull request: https://github.com/apache/drill/pull/1060#discussion_r179321187 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/VLAbstractEntryReader.java --- @@ -0,0 +1,214 @@

[GitHub] drill pull request #1060: DRILL-5846: Improve parquet performance for Flat D...

2018-04-04 Thread vrozov
Github user vrozov commented on a diff in the pull request: https://github.com/apache/drill/pull/1060#discussion_r179296715 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/ParquetRecordReader.java --- @@ -161,30 +162,25 @@ public

[GitHub] drill pull request #1060: DRILL-5846: Improve parquet performance for Flat D...

2018-04-04 Thread sachouche
Github user sachouche commented on a diff in the pull request: https://github.com/apache/drill/pull/1060#discussion_r179296101 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/MemoryUtils.java --- @@ -0,0 +1,199 @@ +/** --- End diff

[GitHub] drill pull request #1060: DRILL-5846: Improve parquet performance for Flat D...

2018-04-04 Thread sachouche
Github user sachouche commented on a diff in the pull request: https://github.com/apache/drill/pull/1060#discussion_r179295602 --- Diff: exec/java-exec/pom.xml --- @@ -836,6 +836,14 @@ org.apache.maven.plugins maven-surefire-plugin +

[GitHub] drill pull request #1060: DRILL-5846: Improve parquet performance for Flat D...

2018-04-04 Thread vrozov
Github user vrozov commented on a diff in the pull request: https://github.com/apache/drill/pull/1060#discussion_r179295162 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/MemoryUtils.java --- @@ -0,0 +1,199 @@ +/** --- End diff --

[GitHub] drill pull request #1060: DRILL-5846: Improve parquet performance for Flat D...

2018-04-04 Thread vrozov
Github user vrozov commented on a diff in the pull request: https://github.com/apache/drill/pull/1060#discussion_r179293968 --- Diff: exec/java-exec/pom.xml --- @@ -836,6 +836,14 @@ org.apache.maven.plugins maven-surefire-plugin +

[GitHub] drill pull request #1060: DRILL-5846: Improve parquet performance for Flat D...

2018-02-02 Thread sachouche
Github user sachouche commented on a diff in the pull request: https://github.com/apache/drill/pull/1060#discussion_r165698800 --- Diff: exec/memory/base/src/main/java/io/netty/buffer/DrillBuf.java --- @@ -703,7 +703,18 @@ protected void _setLong(int index, long value) {

[GitHub] drill pull request #1060: DRILL-5846: Improve parquet performance for Flat D...

2018-02-01 Thread vrozov
Github user vrozov commented on a diff in the pull request: https://github.com/apache/drill/pull/1060#discussion_r165552608 --- Diff: exec/memory/base/src/main/java/io/netty/buffer/DrillBuf.java --- @@ -703,7 +703,18 @@ protected void _setLong(int index, long value) {

[GitHub] drill pull request #1060: DRILL-5846: Improve parquet performance for Flat D...

2018-01-29 Thread vrozov
Github user vrozov commented on a diff in the pull request: https://github.com/apache/drill/pull/1060#discussion_r164600674 --- Diff: exec/memory/base/src/main/java/org/apache/drill/exec/util/MemoryUtils.java --- @@ -0,0 +1,186 @@ +/** + * Licensed to the Apache Software

[GitHub] drill pull request #1060: DRILL-5846: Improve parquet performance for Flat D...

2018-01-29 Thread sachouche
Github user sachouche commented on a diff in the pull request: https://github.com/apache/drill/pull/1060#discussion_r164532290 --- Diff: exec/memory/base/src/main/java/io/netty/buffer/DrillBuf.java --- @@ -703,7 +703,18 @@ protected void _setLong(int index, long value) {

[GitHub] drill pull request #1060: DRILL-5846: Improve parquet performance for Flat D...

2018-01-29 Thread sachouche
Github user sachouche commented on a diff in the pull request: https://github.com/apache/drill/pull/1060#discussion_r164525752 --- Diff: exec/memory/base/src/main/java/org/apache/drill/exec/util/MemoryUtils.java --- @@ -0,0 +1,186 @@ +/** + * Licensed to the Apache

[GitHub] drill pull request #1060: DRILL-5846: Improve parquet performance for Flat D...

2018-01-29 Thread vrozov
Github user vrozov commented on a diff in the pull request: https://github.com/apache/drill/pull/1060#discussion_r164520611 --- Diff: exec/memory/base/src/main/java/io/netty/buffer/DrillBuf.java --- @@ -703,7 +703,18 @@ protected void _setLong(int index, long value) {

[GitHub] drill pull request #1060: DRILL-5846: Improve parquet performance for Flat D...

2018-01-29 Thread vrozov
Github user vrozov commented on a diff in the pull request: https://github.com/apache/drill/pull/1060#discussion_r164519147 --- Diff: exec/memory/base/src/main/java/org/apache/drill/exec/util/MemoryUtils.java --- @@ -0,0 +1,186 @@ +/** + * Licensed to the Apache Software

[GitHub] drill pull request #1060: DRILL-5846: Improve parquet performance for Flat D...

2018-01-29 Thread sachouche
Github user sachouche commented on a diff in the pull request: https://github.com/apache/drill/pull/1060#discussion_r164509658 --- Diff: exec/memory/base/src/main/java/org/apache/drill/exec/util/MemoryUtils.java --- @@ -0,0 +1,186 @@ +/** + * Licensed to the Apache

[GitHub] drill pull request #1060: DRILL-5846: Improve parquet performance for Flat D...

2018-01-29 Thread vrozov
Github user vrozov commented on a diff in the pull request: https://github.com/apache/drill/pull/1060#discussion_r164506523 --- Diff: exec/memory/base/src/main/java/org/apache/drill/exec/util/MemoryUtils.java --- @@ -0,0 +1,186 @@ +/** + * Licensed to the Apache Software

[GitHub] drill pull request #1060: DRILL-5846: Improve parquet performance for Flat D...

2018-01-21 Thread sachouche
Github user sachouche commented on a diff in the pull request: https://github.com/apache/drill/pull/1060#discussion_r162828863 --- Diff: exec/vector/src/main/codegen/templates/NullableValueVectors.java --- @@ -68,96 +85,441 @@ private final UInt1Vector bits = new

[GitHub] drill pull request #1060: DRILL-5846: Improve parquet performance for Flat D...

2018-01-21 Thread sachouche
Github user sachouche commented on a diff in the pull request: https://github.com/apache/drill/pull/1060#discussion_r162828312 --- Diff: exec/vector/src/main/codegen/templates/NullableValueVectors.java --- @@ -51,6 +57,17 @@ public final class ${className} extends

[GitHub] drill pull request #1060: DRILL-5846: Improve parquet performance for Flat D...

2018-01-21 Thread sachouche
Github user sachouche commented on a diff in the pull request: https://github.com/apache/drill/pull/1060#discussion_r162828788 --- Diff: exec/vector/src/main/codegen/templates/NullableValueVectors.java --- @@ -68,96 +85,441 @@ private final UInt1Vector bits = new

[GitHub] drill pull request #1060: DRILL-5846: Improve parquet performance for Flat D...

2018-01-21 Thread sachouche
Github user sachouche commented on a diff in the pull request: https://github.com/apache/drill/pull/1060#discussion_r162828174 --- Diff: exec/vector/src/main/codegen/templates/NullableValueVectors.java --- @@ -51,6 +57,17 @@ public final class ${className} extends

[GitHub] drill pull request #1060: DRILL-5846: Improve parquet performance for Flat D...

2018-01-11 Thread sachouche
Github user sachouche commented on a diff in the pull request: https://github.com/apache/drill/pull/1060#discussion_r161065237 --- Diff: protocol/src/main/protobuf/UserBitShared.proto --- @@ -148,6 +148,8 @@ message SerializedField { optional int32 value_count = 4;

[GitHub] drill pull request #1060: DRILL-5846: Improve parquet performance for Flat D...

2018-01-11 Thread sachouche
Github user sachouche commented on a diff in the pull request: https://github.com/apache/drill/pull/1060#discussion_r161040070 --- Diff: exec/vector/src/main/codegen/templates/VariableLengthVectors.java --- @@ -309,7 +314,7 @@ public void setInitialCapacity(final int valueCount) {

[GitHub] drill pull request #1060: DRILL-5846: Improve parquet performance for Flat D...

2018-01-11 Thread sachouche
Github user sachouche commented on a diff in the pull request: https://github.com/apache/drill/pull/1060#discussion_r161039122 --- Diff: exec/vector/src/main/codegen/templates/FixedValueVectors.java --- @@ -874,6 +880,46 @@ public void setSafe(int index, BigDecimal value) {

[GitHub] drill pull request #1060: DRILL-5846: Improve parquet performance for Flat D...

2018-01-11 Thread sachouche
Github user sachouche commented on a diff in the pull request: https://github.com/apache/drill/pull/1060#discussion_r161036145 --- Diff: exec/memory/base/src/main/java/org/apache/drill/exec/util/MemoryUtils.java --- @@ -0,0 +1,186 @@ +/** + * Licensed to the Apache

[GitHub] drill pull request #1060: DRILL-5846: Improve parquet performance for Flat D...

2018-01-11 Thread sachouche
Github user sachouche commented on a diff in the pull request: https://github.com/apache/drill/pull/1060#discussion_r161045748 --- Diff: exec/vector/src/main/codegen/templates/VariableLengthVectors.java --- @@ -386,7 +391,7 @@ public void reAlloc() { throw new

[GitHub] drill pull request #1060: DRILL-5846: Improve parquet performance for Flat D...

2018-01-11 Thread sachouche
Github user sachouche commented on a diff in the pull request: https://github.com/apache/drill/pull/1060#discussion_r161030963 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/server/options/SystemOptionManager.java --- @@ -449,7 +451,7 @@ public void close() throws

[GitHub] drill pull request #1060: DRILL-5846: Improve parquet performance for Flat D...

2018-01-11 Thread sachouche
Github user sachouche commented on a diff in the pull request: https://github.com/apache/drill/pull/1060#discussion_r161032779 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/NullableColumnReader.java --- @@ -165,17 +181,133 @@

[GitHub] drill pull request #1060: DRILL-5846: Improve parquet performance for Flat D...

2018-01-11 Thread sachouche
Github user sachouche commented on a diff in the pull request: https://github.com/apache/drill/pull/1060#discussion_r161028252 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/ScanBatch.java --- @@ -471,8 +519,8 @@ public void close() throws Exception {

[GitHub] drill pull request #1060: DRILL-5846: Improve parquet performance for Flat D...

2018-01-09 Thread sachouche
Github user sachouche commented on a diff in the pull request: https://github.com/apache/drill/pull/1060#discussion_r160514755 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/VLAbstractEntryReader.java --- @@ -0,0 +1,215 @@

[GitHub] drill pull request #1060: DRILL-5846: Improve parquet performance for Flat D...

2017-12-22 Thread parthchandra
Github user parthchandra commented on a diff in the pull request: https://github.com/apache/drill/pull/1060#discussion_r158549859 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/VLAbstractEntryReader.java --- @@ -0,0 +1,215 @@

[GitHub] drill pull request #1060: DRILL-5846: Improve parquet performance for Flat D...

2017-12-20 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1060#discussion_r158183470 --- Diff: exec/vector/src/main/codegen/templates/VariableLengthVectors.java --- @@ -309,7 +314,7 @@ public void setInitialCapacity(final int valueCount)

[GitHub] drill pull request #1060: DRILL-5846: Improve parquet performance for Flat D...

2017-12-20 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1060#discussion_r158197926 --- Diff: exec/vector/src/main/codegen/templates/NullableValueVectors.java --- @@ -296,26 +655,75 @@ public void transferTo(Nullable${minor.class}Vector

[GitHub] drill pull request #1060: DRILL-5846: Improve parquet performance for Flat D...

2017-12-20 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1060#discussion_r158196904 --- Diff: exec/vector/src/main/codegen/templates/NullableValueVectors.java --- @@ -68,96 +85,441 @@ private final UInt1Vector bits = new

[GitHub] drill pull request #1060: DRILL-5846: Improve parquet performance for Flat D...

2017-12-20 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1060#discussion_r158196570 --- Diff: exec/vector/src/main/codegen/templates/NullableValueVectors.java --- @@ -68,96 +85,441 @@ private final UInt1Vector bits = new

[GitHub] drill pull request #1060: DRILL-5846: Improve parquet performance for Flat D...

2017-12-20 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1060#discussion_r158183647 --- Diff: exec/vector/src/main/codegen/templates/NullableValueVectors.java --- @@ -15,13 +15,19 @@ * See the License for the specific language

[GitHub] drill pull request #1060: DRILL-5846: Improve parquet performance for Flat D...

2017-12-20 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1060#discussion_r158181548 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/ScanBatch.java --- @@ -471,8 +519,8 @@ public void close() throws Exception

[GitHub] drill pull request #1060: DRILL-5846: Improve parquet performance for Flat D...

2017-12-20 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1060#discussion_r158197616 --- Diff: exec/vector/src/main/codegen/templates/FixedValueVectors.java --- @@ -874,6 +880,46 @@ public void setSafe(int index, BigDecimal value) {

[GitHub] drill pull request #1060: DRILL-5846: Improve parquet performance for Flat D...

2017-12-20 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1060#discussion_r158198379 --- Diff: exec/vector/src/main/codegen/templates/NullableValueVectors.java --- @@ -804,6 +1406,310 @@ public void fromNotNullable(${minor.class}Vector

[GitHub] drill pull request #1060: DRILL-5846: Improve parquet performance for Flat D...

2017-12-20 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1060#discussion_r158197380 --- Diff: exec/memory/base/src/main/java/org/apache/drill/exec/util/MemoryUtils.java --- @@ -0,0 +1,186 @@ +/** + * Licensed to the Apache

[GitHub] drill pull request #1060: DRILL-5846: Improve parquet performance for Flat D...

2017-12-20 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1060#discussion_r158198205 --- Diff: exec/vector/src/main/codegen/templates/NullableValueVectors.java --- @@ -456,55 +818,191 @@ public void get(int index,

[GitHub] drill pull request #1060: DRILL-5846: Improve parquet performance for Flat D...

2017-12-20 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1060#discussion_r158183682 --- Diff: exec/vector/src/main/codegen/templates/NullableValueVectors.java --- @@ -15,13 +15,19 @@ * See the License for the specific language

[GitHub] drill pull request #1060: DRILL-5846: Improve parquet performance for Flat D...

2017-12-20 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1060#discussion_r158183275 --- Diff: protocol/src/main/protobuf/UserBitShared.proto --- @@ -148,6 +148,8 @@ message SerializedField { optional int32 value_count = 4;

[GitHub] drill pull request #1060: DRILL-5846: Improve parquet performance for Flat D...

2017-12-20 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1060#discussion_r158197120 --- Diff: exec/vector/src/main/codegen/templates/NullableValueVectors.java --- @@ -68,96 +85,441 @@ private final UInt1Vector bits = new

[GitHub] drill pull request #1060: DRILL-5846: Improve parquet performance for Flat D...

2017-12-20 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1060#discussion_r158196450 --- Diff: exec/vector/src/main/codegen/templates/NullableValueVectors.java --- @@ -68,96 +85,441 @@ private final UInt1Vector bits = new

[GitHub] drill pull request #1060: DRILL-5846: Improve parquet performance for Flat D...

2017-12-20 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1060#discussion_r158196813 --- Diff: exec/vector/src/main/codegen/templates/NullableValueVectors.java --- @@ -51,6 +57,17 @@ public final class ${className} extends

[GitHub] drill pull request #1060: DRILL-5846: Improve parquet performance for Flat D...

2017-12-20 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1060#discussion_r158197511 --- Diff: exec/vector/.externalToolBuilders/Maven_Ant_Builder.launch --- @@ -0,0 +1,25 @@ + --- End diff -- Did this file want to be

[GitHub] drill pull request #1060: DRILL-5846: Improve parquet performance for Flat D...

2017-12-20 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1060#discussion_r158182667 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/NullableColumnReader.java --- @@ -165,17 +181,133 @@

[GitHub] drill pull request #1060: DRILL-5846: Improve parquet performance for Flat D...

2017-12-20 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1060#discussion_r158196388 --- Diff: exec/vector/src/main/codegen/templates/NullableValueVectors.java --- @@ -68,96 +85,441 @@ private final UInt1Vector bits = new

[GitHub] drill pull request #1060: DRILL-5846: Improve parquet performance for Flat D...

2017-12-20 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1060#discussion_r158196333 --- Diff: exec/vector/src/main/codegen/templates/NullableValueVectors.java --- @@ -51,6 +57,17 @@ public final class ${className} extends

[GitHub] drill pull request #1060: DRILL-5846: Improve parquet performance for Flat D...

2017-12-20 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1060#discussion_r158183519 --- Diff: exec/vector/src/main/codegen/templates/VariableLengthVectors.java --- @@ -386,7 +391,7 @@ public void reAlloc() { throw new

[GitHub] drill pull request #1060: DRILL-5846: Improve parquet performance for Flat D...

2017-12-20 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1060#discussion_r158181731 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/server/options/SystemOptionManager.java --- @@ -449,7 +451,7 @@ public void close() throws

[GitHub] drill pull request #1060: DRILL-5846: Improve parquet performance for Flat D...

2017-12-03 Thread sachouche
GitHub user sachouche opened a pull request: https://github.com/apache/drill/pull/1060 DRILL-5846: Improve parquet performance for Flat Data Types Performance improvements for the Parquet Scanner (Flat Data Types). The are two flags to control this performance enhancement (disabled