[GitHub] drill issue #540: Fix for DRILL-4759: Drill throwing array index out of boun...

2016-07-13 Thread ppadma
Github user ppadma commented on the issue: https://github.com/apache/drill/pull/540 updated the pull request with test case added. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] drill issue #540: Fix for DRILL-4759: Drill throwing array index out of boun...

2016-07-18 Thread ppadma
Github user ppadma commented on the issue: https://github.com/apache/drill/pull/540 updated the fix. Please review. In DictionaryBigIntReader, we need to call parent readField if dictionary encoding is not enabled for the page. --- If your project is set up for it, you can reply

[GitHub] drill issue #540: Fix for DRILL-4759: Drill throwing array index out of boun...

2016-07-19 Thread ppadma
Github user ppadma commented on the issue: https://github.com/apache/drill/pull/540 The pull request was merged in commit: e371e18 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] drill pull request #540: Fix for DRILL-4759: Drill throwing array index out ...

2016-07-19 Thread ppadma
Github user ppadma closed the pull request at: https://github.com/apache/drill/pull/540 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] drill pull request #540: Fix for DRILL-4759: Drill throwing array index out ...

2016-07-05 Thread ppadma
GitHub user ppadma opened a pull request: https://github.com/apache/drill/pull/540 Fix for DRILL-4759: Drill throwing array index out of bound exception… … when reading a parquet file written by map reduce program You can merge this pull request into a Git repository by running

[GitHub] drill pull request #540: Fix for DRILL-4759: Drill throwing array index out ...

2016-07-05 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/540#discussion_r69650754 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/ParquetFixedWidthDictionaryReaders.java --- @@ -156,12 +156,18

[GitHub] drill pull request #539: DRILL-4759:Drill throwing array index out of bound ...

2016-07-01 Thread ppadma
GitHub user ppadma opened a pull request: https://github.com/apache/drill/pull/539 DRILL-4759:Drill throwing array index out of bound exception when reading a parquet file written by map reduce program. You can merge this pull request into a Git repository by running: $ git

[GitHub] drill pull request #539: DRILL-4759:Drill throwing array index out of bound ...

2016-07-01 Thread ppadma
Github user ppadma closed the pull request at: https://github.com/apache/drill/pull/539 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] drill pull request #730: DRILL-5223:Drill should ensure balanced workload as...

2017-01-26 Thread ppadma
GitHub user ppadma opened a pull request: https://github.com/apache/drill/pull/730 DRILL-5223:Drill should ensure balanced workload assignment at node l… …evel in order to get better query performance. Please see DRILL-5223 for details: https://issues.apache.org

[GitHub] drill pull request #704: DRILL-5125: Provide option to use generic code for ...

2017-01-27 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/704#discussion_r98299585 --- Diff: exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/svremover/TestSVRemover.java --- @@ -34,4 +38,33 @@ public void testSVRWithNoFilter

[GitHub] drill pull request #704: DRILL-5125: Provide option to use generic code for ...

2017-01-27 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/704#discussion_r97992610 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/svremover/GenericSV4Copier.java --- @@ -0,0 +1,67 @@ +/* + * Licensed

[GitHub] drill pull request #747: DRILL-5257: Run-time control of query profiles

2017-02-21 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/747#discussion_r102378871 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/Foreman.java --- @@ -178,6 +182,19 @@ public Foreman(final WorkerBee bee, final

[GitHub] drill pull request #747: DRILL-5257: Run-time control of query profiles

2017-02-21 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/747#discussion_r102378651 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/ExecConstants.java --- @@ -413,4 +413,15 @@ String DYNAMIC_UDF_SUPPORT_ENABLED

[GitHub] drill pull request #751: DRILL-5259: Allow listing a user-defined number of ...

2017-02-17 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/751#discussion_r101862127 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/server/rest/profile/ProfileResources.java --- @@ -190,10 +193,13 @@ public QProfiles(List

[GitHub] drill pull request #749: DRILL-5266: Parquet returns low-density batches

2017-02-23 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/749#discussion_r102830312 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/VarLenBinaryReader.java --- @@ -33,35 +33,52

[GitHub] drill pull request #749: DRILL-5266: Parquet returns low-density batches

2017-02-23 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/749#discussion_r102827651 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/VarLenBinaryReader.java --- @@ -70,33 +87,21 @@ public long

[GitHub] drill pull request #749: DRILL-5266: Parquet returns low-density batches

2017-02-23 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/749#discussion_r102826142 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/VarLenBinaryReader.java --- @@ -33,35 +33,52

[GitHub] drill pull request #749: DRILL-5266: Parquet returns low-density batches

2017-02-23 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/749#discussion_r102824318 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/ParquetRecordReader.java --- @@ -351,7 +353,7 @@ public void setup

[GitHub] drill issue #757: DRILL-5290: Provide an option to build operator table once...

2017-02-23 Thread ppadma
Github user ppadma commented on the issue: https://github.com/apache/drill/pull/757 Thanks Sudheesh, Paul and Arina for the review. Original thought was it is not worth the effort to keep operator table in sync with changes in dynamic UDFs. This option is meant to be used for short

[GitHub] drill pull request #757: DRILL-5290: Provide an option to build operator tab...

2017-02-22 Thread ppadma
GitHub user ppadma opened a pull request: https://github.com/apache/drill/pull/757 DRILL-5290: Provide an option to build operator table once for built-… …in static functions and reuse it across queries. You can merge this pull request into a Git repository by running

[GitHub] drill pull request #758: DRILL-5287: Provide option to skip updates of ephem...

2017-02-24 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/758#discussion_r103005063 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/Foreman.java --- @@ -1010,7 +1010,9 @@ public void addToEventQueue(final QueryState

[GitHub] drill pull request #749: DRILL-5266: Parquet returns low-density batches

2017-02-23 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/749#discussion_r102725896 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/ParquetRecordReader.java --- @@ -376,14 +378,14 @@ public void setup

[GitHub] drill pull request #749: DRILL-5266: Parquet returns low-density batches

2017-02-23 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/749#discussion_r102726321 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/ParquetRecordReader.java --- @@ -73,6 +72,7 @@ private static

[GitHub] drill pull request #757: DRILL-5290: Provide an option to build operator tab...

2017-02-22 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/757#discussion_r102611281 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/ExecConstants.java --- @@ -413,4 +413,8 @@ String DYNAMIC_UDF_SUPPORT_ENABLED

[GitHub] drill pull request #749: DRILL-5266: Parquet returns low-density batches

2017-02-23 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/749#discussion_r102726705 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/VarLenBinaryReader.java --- @@ -70,33 +70,31 @@ public long

[GitHub] drill pull request #747: DRILL-5257: Run-time control of query profiles

2017-02-21 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/747#discussion_r102277543 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/ExecConstants.java --- @@ -413,4 +413,19 @@ String DYNAMIC_UDF_SUPPORT_ENABLED

[GitHub] drill pull request #747: DRILL-5257: Run-time control of query profiles

2017-02-21 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/747#discussion_r102278999 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/Foreman.java --- @@ -853,7 +875,9 @@ public void close() throws Exception

[GitHub] drill pull request #757: DRILL-5290: Provide an option to build operator tab...

2017-02-22 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/757#discussion_r102615118 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/ExecConstants.java --- @@ -413,4 +413,8 @@ String DYNAMIC_UDF_SUPPORT_ENABLED

[GitHub] drill pull request #758: DRILL-5287: Provide option to skip updates of ephem...

2017-02-28 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/758#discussion_r103487885 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/QueryManager.java --- @@ -280,8 +281,15 @@ public void interrupted(final

[GitHub] drill pull request #766: DRILL-5304: Queries fail intermittently when there ...

2017-02-28 Thread ppadma
GitHub user ppadma opened a pull request: https://github.com/apache/drill/pull/766 DRILL-5304: Queries fail intermittently when there is skew in data di… …stribution Change the assignment logic so we first make sure we assign up to minCount for all nodes before going

[GitHub] drill pull request #757: DRILL-5290: Provide an option to build operator tab...

2017-02-27 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/757#discussion_r103285068 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/ExecConstants.java --- @@ -413,4 +413,8 @@ String DYNAMIC_UDF_SUPPORT_ENABLED

[GitHub] drill pull request #597: DRILL-4905: Push down the LIMIT to the parquet read...

2016-09-26 Thread ppadma
GitHub user ppadma opened a pull request: https://github.com/apache/drill/pull/597 DRILL-4905: Push down the LIMIT to the parquet reader scan. For limit N query, where N is less than current default record batchSize (256K for all fixedlength, 32K otherwise), we still end up reading

[GitHub] drill pull request #597: DRILL-4905: Push down the LIMIT to the parquet read...

2016-09-29 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/597#discussion_r81244611 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetGroupScan.java --- @@ -864,6 +872,14 @@ public String getDigest

[GitHub] drill pull request #597: DRILL-4905: Push down the LIMIT to the parquet read...

2016-09-28 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/597#discussion_r80990803 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetGroupScan.java --- @@ -115,6 +115,8 @@ private List rowGroupInfos

[GitHub] drill pull request #597: DRILL-4905: Push down the LIMIT to the parquet read...

2016-09-27 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/597#discussion_r80817027 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetGroupScan.java --- @@ -115,6 +115,8 @@ private List rowGroupInfos

[GitHub] drill pull request #597: DRILL-4905: Push down the LIMIT to the parquet read...

2016-09-27 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/597#discussion_r80754060 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetGroupScan.java --- @@ -899,6 +907,16 @@ public FileGroupScan clone

[GitHub] drill pull request #597: DRILL-4905: Push down the LIMIT to the parquet read...

2016-09-27 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/597#discussion_r80751210 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetScanBatchCreator.java --- @@ -107,7 +107,7 @@ public ScanBatch getBatch

[GitHub] drill issue #597: DRILL-4905: Push down the LIMIT to the parquet reader scan...

2016-09-27 Thread ppadma
Github user ppadma commented on the issue: https://github.com/apache/drill/pull/597 updated with review comments. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] drill pull request #597: DRILL-4905: Push down the LIMIT to the parquet read...

2016-09-27 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/597#discussion_r80788614 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetGroupScan.java --- @@ -899,6 +907,16 @@ public FileGroupScan clone

[GitHub] drill pull request #597: DRILL-4905: Push down the LIMIT to the parquet read...

2016-09-27 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/597#discussion_r80788617 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetGroupScan.java --- @@ -899,6 +907,16 @@ public FileGroupScan clone

[GitHub] drill pull request #597: DRILL-4905: Push down the LIMIT to the parquet read...

2016-09-27 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/597#discussion_r80788478 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetScanBatchCreator.java --- @@ -107,7 +107,7 @@ public ScanBatch getBatch

[GitHub] drill pull request #597: DRILL-4905: Push down the LIMIT to the parquet read...

2016-09-29 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/597#discussion_r81238087 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetGroupScan.java --- @@ -864,6 +872,14 @@ public String getDigest

[GitHub] drill issue #597: DRILL-4905: Push down the LIMIT to the parquet reader scan...

2016-09-29 Thread ppadma
Github user ppadma commented on the issue: https://github.com/apache/drill/pull/597 updated with all review comments taken care of. Please review. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] drill pull request #611: Drill-4800: Improve parquet reader performance

2016-10-26 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/611#discussion_r84952909 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/util/filereader/BufferedDirectBufInputStream.java --- @@ -0,0 +1,467 @@ +/** + * Licensed

[GitHub] drill pull request #611: Drill-4800: Improve parquet reader performance

2016-10-26 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/611#discussion_r85028094 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/VarLenBinaryReader.java --- @@ -41,43 +51,149 @@ public

[GitHub] drill pull request #611: Drill-4800: Improve parquet reader performance

2016-10-26 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/611#discussion_r84972962 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/AsyncPageReader.java --- @@ -0,0 +1,409 @@ +/** + * Licensed

[GitHub] drill pull request #611: Drill-4800: Improve parquet reader performance

2016-10-26 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/611#discussion_r85136996 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/ops/OperatorContextImpl.java --- @@ -174,4 +191,11 @@ public DrillFileSystem newFileSystem

[GitHub] drill pull request #611: Drill-4800: Improve parquet reader performance

2016-10-26 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/611#discussion_r84972161 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/AsyncPageReader.java --- @@ -0,0 +1,409 @@ +/** + * Licensed

[GitHub] drill pull request #611: Drill-4800: Improve parquet reader performance

2016-10-26 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/611#discussion_r85141426 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/AsyncPageReader.java --- @@ -0,0 +1,409 @@ +/** + * Licensed

[GitHub] drill pull request #611: Drill-4800: Improve parquet reader performance

2016-10-26 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/611#discussion_r84981300 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/AsyncPageReader.java --- @@ -0,0 +1,409 @@ +/** + * Licensed

[GitHub] drill pull request #611: Drill-4800: Improve parquet reader performance

2016-10-26 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/611#discussion_r84957411 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/PageReader.java --- @@ -99,26 +99,47 @@ int

[GitHub] drill pull request #611: Drill-4800: Improve parquet reader performance

2016-10-26 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/611#discussion_r84970035 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/ColumnReader.java --- @@ -148,7 +185,8 @@ public boolean

[GitHub] drill pull request #611: Drill-4800: Improve parquet reader performance

2016-10-26 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/611#discussion_r84968348 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/AsyncPageReader.java --- @@ -0,0 +1,409 @@ +/** + * Licensed

[GitHub] drill pull request #611: Drill-4800: Improve parquet reader performance

2016-10-26 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/611#discussion_r84969218 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/AsyncPageReader.java --- @@ -0,0 +1,409 @@ +/** + * Licensed

[GitHub] drill pull request #611: Drill-4800: Improve parquet reader performance

2016-10-26 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/611#discussion_r84970233 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/ops/OperatorContextImpl.java --- @@ -174,4 +191,11 @@ public DrillFileSystem newFileSystem

[GitHub] drill pull request #611: Drill-4800: Improve parquet reader performance

2016-10-26 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/611#discussion_r85152503 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/util/filereader/BufferedDirectBufInputStream.java --- @@ -0,0 +1,467 @@ +/** + * Licensed

[GitHub] drill pull request #611: Drill-4800: Improve parquet reader performance

2016-10-26 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/611#discussion_r84971152 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/AsyncPageReader.java --- @@ -0,0 +1,409 @@ +/** + * Licensed

[GitHub] drill pull request #611: Drill-4800: Improve parquet reader performance

2016-10-26 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/611#discussion_r85153296 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/util/filereader/DirectBufInputStream.java --- @@ -0,0 +1,183 @@ +/** + * Licensed

[GitHub] drill pull request #611: Drill-4800: Improve parquet reader performance

2016-10-26 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/611#discussion_r84990242 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/AsyncPageReader.java --- @@ -0,0 +1,409 @@ +/** + * Licensed

[GitHub] drill pull request #611: Drill-4800: Improve parquet reader performance

2016-10-26 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/611#discussion_r84953931 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/AsyncPageReader.java --- @@ -0,0 +1,409 @@ +/** + * Licensed

[GitHub] drill pull request #611: Drill-4800: Improve parquet reader performance

2016-10-26 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/611#discussion_r85146088 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/AsyncPageReader.java --- @@ -0,0 +1,409 @@ +/** + * Licensed

[GitHub] drill pull request #611: Drill-4800: Improve parquet reader performance

2016-10-26 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/611#discussion_r85150281 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/ColumnReader.java --- @@ -234,4 +286,48 @@ public static int

[GitHub] drill pull request #611: Drill-4800: Improve parquet reader performance

2016-10-26 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/611#discussion_r84987279 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/server/BootStrapContext.java --- @@ -45,12 +46,15 @@ private final BufferAllocator

[GitHub] drill pull request #611: Drill-4800: Improve parquet reader performance

2016-10-26 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/611#discussion_r85152665 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/util/filereader/BufferedDirectBufInputStream.java --- @@ -0,0 +1,467 @@ +/** + * Licensed

[GitHub] drill pull request #611: Drill-4800: Improve parquet reader performance

2016-10-26 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/611#discussion_r84982984 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/ops/OperatorContext.java --- @@ -43,10 +45,18 @@ public abstract OperatorStats

[GitHub] drill pull request #611: Drill-4800: Improve parquet reader performance

2016-10-26 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/611#discussion_r85153076 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/util/filereader/DirectBufInputStream.java --- @@ -0,0 +1,183 @@ +/** + * Licensed

[GitHub] drill pull request #611: Drill-4800: Improve parquet reader performance

2016-10-26 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/611#discussion_r85153504 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/util/filereader/DirectBufInputStream.java --- @@ -0,0 +1,183 @@ +/** + * Licensed

[GitHub] drill issue #639: DRILL-4706: Fragment planning causes Drillbits to read rem...

2016-11-09 Thread ppadma
Github user ppadma commented on the issue: https://github.com/apache/drill/pull/639 Merged with latest code. All review comments taken care of. All tests pass with the option `store.parquet.use_local_affinity` = true and false, both. --- If your project is set up for it, you can

[GitHub] drill issue #639: DRILL-4706: Fragment planning causes Drillbits to read rem...

2016-11-04 Thread ppadma
Github user ppadma commented on the issue: https://github.com/apache/drill/pull/639 Parallelization logic is affected for following reasons: Depending upon how many rowGroups to scan on a node (based on locality information) i.e. how much work the node has to do, we want to adjust

[GitHub] drill issue #597: DRILL-4905: Push the LIMIT down to the parquet reader.

2016-10-18 Thread ppadma
Github user ppadma commented on the issue: https://github.com/apache/drill/pull/597 updated diffs with review comments taken care of. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] drill pull request #597: DRILL-4905: Push the LIMIT down to the parquet read...

2016-10-18 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/597#discussion_r83965044 --- Diff: exec/java-exec/src/test/java/org/apache/drill/exec/store/parquet/TestParquetGroupScan.java --- @@ -117,4 +119,18 @@ public void

[GitHub] drill pull request #597: DRILL-4905: Push the LIMIT down to the parquet read...

2016-10-18 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/597#discussion_r83965017 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/ParquetRecordReader.java --- @@ -139,6 +155,11 @@ public

[GitHub] drill pull request #652: DRILL-4990:Use new HDFS API access instead of listS...

2016-11-14 Thread ppadma
GitHub user ppadma opened a pull request: https://github.com/apache/drill/pull/652 DRILL-4990:Use new HDFS API access instead of listStatus to check if … …users have permissions to access workspace. Manually tested the fix with impersonation enabled. All unit

[GitHub] drill issue #652: DRILL-4990:Use new HDFS API access instead of listStatus t...

2016-11-15 Thread ppadma
Github user ppadma commented on the issue: https://github.com/apache/drill/pull/652 I did not add new unit tests because existing tests already provide enough coverage and they run on local file system. --- If your project is set up for it, you can reply to this email and have your

[GitHub] drill issue #653: DRILL-4831: Running refresh table metadata concurrently ra...

2016-11-16 Thread ppadma
Github user ppadma commented on the issue: https://github.com/apache/drill/pull/653 Thanks Gautam for the review. Updated with all review comments addressed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] drill pull request #653: DRILL-4831: Running refresh table metadata concurre...

2016-11-16 Thread ppadma
GitHub user ppadma opened a pull request: https://github.com/apache/drill/pull/653 DRILL-4831: Running refresh table metadata concurrently randomly fail… …s with JsonParseException To prevent metadata cache files from getting corrupted, write to a temporary file and do

[GitHub] drill issue #653: DRILL-4831: Running refresh table metadata concurrently ra...

2016-11-18 Thread ppadma
Github user ppadma commented on the issue: https://github.com/apache/drill/pull/653 Updated diffs with all review comments taken care of. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] drill pull request #653: DRILL-4831: Running refresh table metadata concurre...

2016-11-18 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/653#discussion_r88770685 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/Metadata.java --- @@ -495,31 +499,75 @@ private ParquetFileMetadata_v3

[GitHub] drill pull request #652: DRILL-4990:Use new HDFS API access instead of listS...

2016-11-15 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/652#discussion_r88097089 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/dfs/WorkspaceSchemaFactory.java --- @@ -151,15 +152,11 @@ public WorkspaceSchemaFactory

[GitHub] drill issue #639: DRILL-4706: Fragment planning causes Drillbits to read rem...

2016-10-31 Thread ppadma
Github user ppadma commented on the issue: https://github.com/apache/drill/pull/639 Updated the JIRA with details on how current algorithm works, why remote reads were happening and the new algorithm details. https://issues.apache.org/jira/browse/DRILL-4706 --- If your project

[GitHub] drill pull request #639: New fragment placement algorithm based on locality ...

2016-10-31 Thread ppadma
GitHub user ppadma opened a pull request: https://github.com/apache/drill/pull/639 New fragment placement algorithm based on locality of data. You can merge this pull request into a Git repository by running: $ git pull https://github.com/ppadma/drill DRILL-4706

[GitHub] drill issue #639: DRILL-4706: Fragment planning causes Drillbits to read rem...

2016-11-04 Thread ppadma
Github user ppadma commented on the issue: https://github.com/apache/drill/pull/639 Some initial comments. The issue is regarding assigning fragments based on strict locality. So why is the parallelization logic affected, and not exclusively locality? Parallelization

[GitHub] drill issue #639: DRILL-4706: Fragment planning causes Drillbits to read rem...

2016-11-04 Thread ppadma
Github user ppadma commented on the issue: https://github.com/apache/drill/pull/639 Updated with all review comments taken care of. Added TestLocalAffinityFragmentParallelizer.java which has bunch of test cases with examples. --- If your project is set up for it, you can reply

[GitHub] drill pull request #639: DRILL-4706: Fragment planning causes Drillbits to r...

2016-11-04 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/639#discussion_r86597707 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/LocalAffinityFragmentParallelizer.java --- @@ -0,0 +1,149

[GitHub] drill pull request #592: DRILL-4826: Query against INFORMATION_SCHEMA.TABLES...

2016-10-13 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/592#discussion_r83324461 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/ischema/InfoSchemaRecordGenerator.java --- @@ -290,28 +290,30 @@ public Tables

[GitHub] drill issue #597: DRILL-4905: Push down the LIMIT to the parquet reader scan...

2016-10-11 Thread ppadma
Github user ppadma commented on the issue: https://github.com/apache/drill/pull/597 updated with new diffs. Please review. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] drill issue #692: DRILL-5123: Write query profile after sending final respon...

2016-12-16 Thread ppadma
Github user ppadma commented on the issue: https://github.com/apache/drill/pull/692 +1. LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] drill issue #676: DRILL-5091: JDBC unit test fail on Java 8

2016-12-07 Thread ppadma
Github user ppadma commented on the issue: https://github.com/apache/drill/pull/676 +1. Thanks for fixing this and for some more cleanup. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] drill pull request #676: DRILL-5091: JDBC unit test fail on Java 8

2016-12-07 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/676#discussion_r91422697 --- Diff: exec/jdbc/src/test/java/org/apache/drill/jdbc/test/Drill2769UnsupportedReportsUseSqlExceptionTest.java --- @@ -368,10 +381,19 @@ public void

[GitHub] drill pull request #676: DRILL-5091: JDBC unit test fail on Java 8

2016-12-07 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/676#discussion_r91422620 --- Diff: exec/jdbc/src/test/java/org/apache/drill/jdbc/test/Drill2489CallsAfterCloseThrowExceptionsTest.java --- @@ -587,6 +618,12 @@ else

[GitHub] drill pull request #676: DRILL-5091: JDBC unit test fail on Java 8

2016-12-07 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/676#discussion_r91422118 --- Diff: exec/jdbc/src/test/java/org/apache/drill/jdbc/test/Drill2489CallsAfterCloseThrowExceptionsTest.java --- @@ -477,18 +477,32 @@ public void

[GitHub] drill pull request #676: DRILL-5091: JDBC unit test fail on Java 8

2016-12-07 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/676#discussion_r91422511 --- Diff: exec/jdbc/src/test/java/org/apache/drill/jdbc/test/Drill2489CallsAfterCloseThrowExceptionsTest.java --- @@ -543,6 +566,14 @@ else

[GitHub] drill pull request #676: DRILL-5091: JDBC unit test fail on Java 8

2016-12-07 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/676#discussion_r91421638 --- Diff: exec/jdbc/src/test/java/org/apache/drill/jdbc/test/JdbcAssert.java --- @@ -66,6 +66,11 @@ public static void setFactory(ConnectionFactory factory

[GitHub] drill pull request #676: DRILL-5091: JDBC unit test fail on Java 8

2016-12-07 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/676#discussion_r91423240 --- Diff: exec/jdbc/src/test/java/org/apache/drill/jdbc/test/Drill2769UnsupportedReportsUseSqlExceptionTest.java --- @@ -280,6 +280,10 @@ else

[GitHub] drill pull request #676: DRILL-5091: JDBC unit test fail on Java 8

2016-12-07 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/676#discussion_r91421707 --- Diff: exec/jdbc/src/test/java/org/apache/drill/jdbc/test/Drill2769UnsupportedReportsUseSqlExceptionTest.java --- @@ -397,10 +419,19 @@ public void

[GitHub] drill pull request #676: DRILL-5091: JDBC unit test fail on Java 8

2016-12-07 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/676#discussion_r91421952 --- Diff: exec/jdbc/src/test/java/org/apache/drill/jdbc/test/Drill2489CallsAfterCloseThrowExceptionsTest.java --- @@ -477,18 +477,32 @@ public void

[GitHub] drill pull request #704: DRILL-5125: Provide option to use generic code for ...

2017-03-21 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/704#discussion_r107219761 --- Diff: exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/svremover/TestSVRemover.java --- @@ -34,4 +38,33 @@ public void

[GitHub] drill pull request #789: DRILL-5356: Refactor Parquet Record Reader

2017-03-29 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/789#discussion_r108693574 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/BatchReader.java --- @@ -0,0 +1,164 @@ +/* + * Licensed

[GitHub] drill pull request #789: DRILL-5356: Refactor Parquet Record Reader

2017-03-29 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/789#discussion_r108557760 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/ParquetRecordReader.java --- @@ -307,164 +231,49 @@ public

[GitHub] drill pull request #789: DRILL-5356: Refactor Parquet Record Reader

2017-03-29 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/789#discussion_r108667588 --- Diff: exec/java-exec/src/test/java/org/apache/drill/exec/store/parquet/ParquetInternalsTest.java --- @@ -0,0 +1,154 @@ +/* + * Licensed

  1   2   3   4   >