[jira] [Created] (DRILL-5818) Querying from system options is broken

2017-09-25 Thread Parth Chandra (JIRA)
Parth Chandra created DRILL-5818: Summary: Querying from system options is broken Key: DRILL-5818 URL: https://issues.apache.org/jira/browse/DRILL-5818 Project: Apache Drill Issue Type: Bug

Re: Retrieve Column Types from Parquet File?

2017-09-25 Thread Boaz Ben-Zvi
Hi Max, Not via Drill, but you can find the data types of Parquet file(s) by using the parquet-tools (see https://github.com/apache/parquet-mr/tree/master/parquet-tools ) See example below, -- Boaz $ cd ~/parquet-mr/parquet-tools $ java -jar

[GitHub] drill pull request #938: DRILL-5694: Handle HashAgg OOM by spill and retry, ...

2017-09-25 Thread Ben-Zvi
Github user Ben-Zvi closed the pull request at: https://github.com/apache/drill/pull/938 ---

[GitHub] drill pull request #953: DRILL-5259: Allow listing a user-defined number of ...

2017-09-25 Thread kkhatua
Github user kkhatua commented on a diff in the pull request: https://github.com/apache/drill/pull/953#discussion_r140927942 --- Diff: exec/java-exec/src/main/resources/rest/profile/list.ftl --- @@ -37,7 +37,15 @@ No running queries. - Completed

[GitHub] drill pull request #958: DRILL-5808: Reduce memory allocator strictness for ...

2017-09-25 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/958#discussion_r140939339 --- Diff: exec/memory/base/src/main/java/org/apache/drill/exec/memory/Accountant.java --- @@ -22,16 +22,55 @@ import

[GitHub] drill pull request #958: DRILL-5808: Reduce memory allocator strictness for ...

2017-09-25 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/958#discussion_r140939608 --- Diff: exec/memory/base/src/main/java/org/apache/drill/exec/memory/Accountant.java --- @@ -80,6 +119,23 @@ public Accountant(Accountant parent, long

[GitHub] drill pull request #958: DRILL-5808: Reduce memory allocator strictness for ...

2017-09-25 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/958#discussion_r140939480 --- Diff: exec/memory/base/src/main/java/org/apache/drill/exec/memory/Accountant.java --- @@ -22,16 +22,55 @@ import

[GitHub] drill issue #959: DRILL-5816: Hash function produces skewed results on Strin...

2017-09-25 Thread paul-rogers
Github user paul-rogers commented on the issue: https://github.com/apache/drill/pull/959 The change looks benign. I wonder; do we have a test case that exercises the hash functions so we can be certain that the fix actually solves the original problem? ---

[GitHub] drill issue #957: DRILL-5809 Fixed Option Storage Backward Compatibility Iss...

2017-09-25 Thread paul-rogers
Github user paul-rogers commented on the issue: https://github.com/apache/drill/pull/957 Thanks for the changes. +1 ---

[GitHub] drill pull request #958: DRILL-5808: Reduce memory allocator strictness for ...

2017-09-25 Thread Ben-Zvi
Github user Ben-Zvi commented on a diff in the pull request: https://github.com/apache/drill/pull/958#discussion_r140870691 --- Diff: exec/memory/base/src/main/java/org/apache/drill/exec/memory/Accountant.java --- @@ -22,16 +22,55 @@ import

[GitHub] drill pull request #958: DRILL-5808: Reduce memory allocator strictness for ...

2017-09-25 Thread Ben-Zvi
Github user Ben-Zvi commented on a diff in the pull request: https://github.com/apache/drill/pull/958#discussion_r140869257 --- Diff: exec/memory/base/src/main/java/org/apache/drill/exec/memory/Accountant.java --- @@ -22,16 +22,55 @@ import

[GitHub] drill pull request #958: DRILL-5808: Reduce memory allocator strictness for ...

2017-09-25 Thread Ben-Zvi
Github user Ben-Zvi commented on a diff in the pull request: https://github.com/apache/drill/pull/958#discussion_r140938324 --- Diff: exec/memory/base/src/main/java/org/apache/drill/exec/memory/Accountant.java --- @@ -80,6 +119,23 @@ public Accountant(Accountant parent, long

[jira] [Created] (DRILL-5819) Default value of security.admin.user_groups and security.admin.users is "true"

2017-09-25 Thread Prasad Nagaraj Subramanya (JIRA)
Prasad Nagaraj Subramanya created DRILL-5819: Summary: Default value of security.admin.user_groups and security.admin.users is "true" Key: DRILL-5819 URL:

[GitHub] drill pull request #957: DRILL-5809 Fixed Option Storage Backward Compatibil...

2017-09-25 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/957#discussion_r140936258 --- Diff: common/src/test/java/org/apache/drill/testutils/DirTestWatcher.java --- @@ -0,0 +1,61 @@ +/** + * Licensed to the Apache Software

[GitHub] drill pull request #957: DRILL-5809 Fixed Option Storage Backward Compatibil...

2017-09-25 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/957#discussion_r140935951 --- Diff: common/src/test/java/org/apache/drill/testutils/DirTestWatcher.java --- @@ -0,0 +1,61 @@ +/** --- End diff -- For copyright,

[GitHub] drill pull request #957: DRILL-5809 Fixed Option Storage Backward Compatibil...

2017-09-25 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/957#discussion_r140936230 --- Diff: common/src/test/java/org/apache/drill/testutils/DirTestWatcher.java --- @@ -0,0 +1,61 @@ +/** + * Licensed to the Apache Software

[GitHub] drill pull request #957: DRILL-5809 Fixed Option Storage Backward Compatibil...

2017-09-25 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/957#discussion_r140936344 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/server/options/PersistedOptionValue.java --- @@ -0,0 +1,310 @@ +/** + * Licensed to

[GitHub] drill pull request #957: DRILL-5809 Fixed Option Storage Backward Compatibil...

2017-09-25 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/957#discussion_r140936539 --- Diff: exec/java-exec/src/test/java/org/apache/drill/exec/server/options/OptionValueTest.java --- @@ -0,0 +1,72 @@ +/** --- End diff --

[GitHub] drill pull request #957: DRILL-5809 Fixed Option Storage Backward Compatibil...

2017-09-25 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/957#discussion_r140936016 --- Diff: common/src/test/java/org/apache/drill/testutils/DirTestWatcher.java --- @@ -0,0 +1,61 @@ +/** + * Licensed to the Apache Software

[GitHub] drill pull request #957: DRILL-5809 Fixed Option Storage Backward Compatibil...

2017-09-25 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/957#discussion_r140936558 --- Diff: exec/java-exec/src/test/java/org/apache/drill/exec/server/options/PersistedOptionValueTest.java --- @@ -0,0 +1,226 @@ +/** --- End

[GitHub] drill pull request #957: DRILL-5809 Fixed Option Storage Backward Compatibil...

2017-09-25 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/957#discussion_r140936599 --- Diff: exec/java-exec/src/test/java/org/apache/drill/exec/server/options/PersistedOptionValueTest.java --- @@ -0,0 +1,226 @@ +/** + *

[GitHub] drill pull request #957: DRILL-5809 Fixed Option Storage Backward Compatibil...

2017-09-25 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/957#discussion_r140936303 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/server/options/PersistedOptionValue.java --- @@ -0,0 +1,310 @@ +/** --- End diff --

[GitHub] drill pull request #960: DRILL-5815: Option to set query memory as percent o...

2017-09-25 Thread paul-rogers
GitHub user paul-rogers opened a pull request: https://github.com/apache/drill/pull/960 DRILL-5815: Option to set query memory as percent of total This PR provides an alternative way to set the memory per query as a percent of system memory. ### Background Drill

[GitHub] drill issue #957: DRILL-5809 Fixed Option Storage Backward Compatibility Iss...

2017-09-25 Thread ilooner
Github user ilooner commented on the issue: https://github.com/apache/drill/pull/957 @paul-rogers Applied comments and did a few other changes. Unit tests cover backward compatability. I manually tested forward compatability locally by saving some options with the new version using

[GitHub] drill pull request #954: DRILL-5803: Show the hostname for each minor fragme...

2017-09-25 Thread kkhatua
Github user kkhatua commented on a diff in the pull request: https://github.com/apache/drill/pull/954#discussion_r140957231 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/server/rest/profile/ProfileWrapper.java --- @@ -75,24 +75,31 @@ public ProfileWrapper(final

Food for thought about intra-document operation

2017-09-25 Thread Damien Profeta
Hello, A few format handled by Drill enable to work with document, meaning nested and repeated structure instead of just tables. Json and Parquet are the two that come to my mind right now. Document modeling is a great way to express complex object and is used a lot in my company. Drill is

[GitHub] drill pull request #942: DRILL-5781: Fix unit test failures to use tests con...

2017-09-25 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/drill/pull/942 ---

[GitHub] drill issue #957: DRILL-5809 Fixed Option Storage Backward Compatibility Iss...

2017-09-25 Thread ilooner
Github user ilooner commented on the issue: https://github.com/apache/drill/pull/957 @paul-rogers Applied comments and squashed commits ---

[GitHub] drill issue #805: Drill-4139: Exception while trying to prune partition. jav...

2017-09-25 Thread paul-rogers
Github user paul-rogers commented on the issue: https://github.com/apache/drill/pull/805 I'm probably confused. Our goal with the Parquet metadata versioning was that new Drill versions can read old files. Here we have a case in which I was running with old versions of the file,

[GitHub] drill pull request #948: DRILL-5745: Corrected 'location' information in Dri...

2017-09-25 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/drill/pull/948 ---

[GitHub] drill pull request #951: DRILL-5727: Update release profile to generate SHA-...

2017-09-25 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/drill/pull/951 ---

[GitHub] drill pull request #952: DRILL-5438: Amazon S3 bucket can't be queried direc...

2017-09-25 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/drill/pull/952 ---

[GitHub] drill pull request #956: DRILL-5812: Restore the minor fragment Gantt chart ...

2017-09-25 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/drill/pull/956 ---

[jira] [Created] (DRILL-5817) deprecated sys options should be deleted from old zookeeper setting

2017-09-25 Thread Dechang Gu (JIRA)
Dechang Gu created DRILL-5817: - Summary: deprecated sys options should be deleted from old zookeeper setting Key: DRILL-5817 URL: https://issues.apache.org/jira/browse/DRILL-5817 Project: Apache Drill

[GitHub] drill pull request #950: DRILL-5431: SSL Support

2017-09-25 Thread parthchandra
Github user parthchandra commented on a diff in the pull request: https://github.com/apache/drill/pull/950#discussion_r140608922 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/ssl/SSLConfig.java --- @@ -0,0 +1,325 @@ +/* + * Licensed to the Apache Software

[GitHub] drill pull request #950: DRILL-5431: SSL Support

2017-09-25 Thread parthchandra
Github user parthchandra commented on a diff in the pull request: https://github.com/apache/drill/pull/950#discussion_r140605580 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/rpc/user/UserServer.java --- @@ -70,22 +78,80 @@ private static final

[GitHub] drill pull request #950: DRILL-5431: SSL Support

2017-09-25 Thread parthchandra
Github user parthchandra commented on a diff in the pull request: https://github.com/apache/drill/pull/950#discussion_r140621612 --- Diff: exec/rpc/src/main/java/org/apache/drill/exec/rpc/BasicClient.java --- @@ -100,6 +103,8 @@ protected void initChannel(SocketChannel ch) throws

[GitHub] drill pull request #950: DRILL-5431: SSL Support

2017-09-25 Thread parthchandra
Github user parthchandra commented on a diff in the pull request: https://github.com/apache/drill/pull/950#discussion_r140601290 --- Diff: contrib/native/client/readme.linux --- @@ -84,6 +84,21 @@ OR ln -svf libboost_filesystem.a libboost_filesystem-mt.a ln -svf

[GitHub] drill pull request #950: DRILL-5431: SSL Support

2017-09-25 Thread parthchandra
Github user parthchandra commented on a diff in the pull request: https://github.com/apache/drill/pull/950#discussion_r140603757 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/rpc/user/UserConnectionConfig.java --- @@ -75,10 +76,20 @@ } else {

[GitHub] drill pull request #950: DRILL-5431: SSL Support

2017-09-25 Thread parthchandra
Github user parthchandra commented on a diff in the pull request: https://github.com/apache/drill/pull/950#discussion_r140612371 --- Diff: exec/jdbc/src/main/java/org/apache/drill/jdbc/impl/DrillConnectionImpl.java --- @@ -140,12 +140,12 @@ protected DrillConnectionImpl(DriverImpl

[GitHub] drill issue #946: DRILL-5799: native-client: Support alternative build direc...

2017-09-25 Thread parthchandra
Github user parthchandra commented on the issue: https://github.com/apache/drill/pull/946 +1. Thanks for the fix. ---

[GitHub] drill pull request #950: DRILL-5431: SSL Support

2017-09-25 Thread parthchandra
Github user parthchandra commented on a diff in the pull request: https://github.com/apache/drill/pull/950#discussion_r140608117 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/ssl/SSLConfig.java --- @@ -0,0 +1,325 @@ +/* + * Licensed to the Apache Software

[GitHub] drill pull request #950: DRILL-5431: SSL Support

2017-09-25 Thread parthchandra
Github user parthchandra commented on a diff in the pull request: https://github.com/apache/drill/pull/950#discussion_r140601254 --- Diff: common/src/main/java/org/apache/drill/common/config/DrillConfig.java --- @@ -110,8 +110,8 @@ public static DrillConfig create() { *

[GitHub] drill pull request #950: DRILL-5431: SSL Support

2017-09-25 Thread parthchandra
Github user parthchandra commented on a diff in the pull request: https://github.com/apache/drill/pull/950#discussion_r140602937 --- Diff: distribution/src/resources/drill-override-example.conf --- @@ -222,7 +222,35 @@ drill.exec: { # Full workspace name should be indicated

[GitHub] drill pull request #950: DRILL-5431: SSL Support

2017-09-25 Thread parthchandra
Github user parthchandra commented on a diff in the pull request: https://github.com/apache/drill/pull/950#discussion_r140608891 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/ssl/SSLConfig.java --- @@ -0,0 +1,325 @@ +/* + * Licensed to the Apache Software

[GitHub] drill pull request #950: DRILL-5431: SSL Support

2017-09-25 Thread parthchandra
Github user parthchandra commented on a diff in the pull request: https://github.com/apache/drill/pull/950#discussion_r140612241 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/ssl/SSLConfigClient.java --- @@ -0,0 +1,229 @@ +/* + * Licensed to the Apache

[GitHub] drill pull request #950: DRILL-5431: SSL Support

2017-09-25 Thread parthchandra
Github user parthchandra commented on a diff in the pull request: https://github.com/apache/drill/pull/950#discussion_r140602783 --- Diff: distribution/src/resources/drill-override-example.conf --- @@ -222,7 +222,35 @@ drill.exec: { # Full workspace name should be indicated

[GitHub] drill pull request #950: DRILL-5431: SSL Support

2017-09-25 Thread parthchandra
Github user parthchandra commented on a diff in the pull request: https://github.com/apache/drill/pull/950#discussion_r140611290 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/ssl/SSLConfigBuilder.java --- @@ -0,0 +1,86 @@ +/* + * Licensed to the Apache

[GitHub] drill pull request #950: DRILL-5431: SSL Support

2017-09-25 Thread parthchandra
Github user parthchandra commented on a diff in the pull request: https://github.com/apache/drill/pull/950#discussion_r140621780 --- Diff: exec/rpc/src/main/java/org/apache/drill/exec/rpc/BasicClient.java --- @@ -120,6 +125,25 @@ protected void initChannel(SocketChannel ch) throws

[GitHub] drill pull request #950: DRILL-5431: SSL Support

2017-09-25 Thread parthchandra
Github user parthchandra commented on a diff in the pull request: https://github.com/apache/drill/pull/950#discussion_r140610830 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/ssl/SSLConfig.java --- @@ -0,0 +1,325 @@ +/* + * Licensed to the Apache Software

[GitHub] drill pull request #950: DRILL-5431: SSL Support

2017-09-25 Thread parthchandra
Github user parthchandra commented on a diff in the pull request: https://github.com/apache/drill/pull/950#discussion_r140603400 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/ExecConstants.java --- @@ -126,10 +138,10 @@ String HTTP_SESSION_MEMORY_RESERVATION

[GitHub] drill pull request #950: DRILL-5431: SSL Support

2017-09-25 Thread parthchandra
Github user parthchandra commented on a diff in the pull request: https://github.com/apache/drill/pull/950#discussion_r140621875 --- Diff: exec/rpc/src/main/java/org/apache/drill/exec/rpc/BasicServer.java --- @@ -203,6 +226,9 @@ public void close() throws IOException {

[GitHub] drill pull request #950: DRILL-5431: SSL Support

2017-09-25 Thread parthchandra
Github user parthchandra commented on a diff in the pull request: https://github.com/apache/drill/pull/950#discussion_r140603240 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/ExecConstants.java --- @@ -110,6 +110,18 @@ String HASHAGG_FALLBACK_ENABLED_KEY =

[GitHub] drill pull request #950: DRILL-5431: SSL Support

2017-09-25 Thread parthchandra
Github user parthchandra commented on a diff in the pull request: https://github.com/apache/drill/pull/950#discussion_r140611274 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/ssl/SSLConfigBuilder.java --- @@ -0,0 +1,86 @@ +/* + * Licensed to the Apache

Re: Food for thought about intra-document operation

2017-09-25 Thread Aman Sinha
Damien, thanks for initiating the discussion..indeed this would be a very useful enhancement. Currently, Drill provides repeated_contains() for filtering and repeated_count() for count aggregates on arrays but not the general purpose intra-document operations that you need based on your example.

[GitHub] drill issue #805: Drill-4139: Exception while trying to prune partition. jav...

2017-09-25 Thread vvysotskyi
Github user vvysotskyi commented on the issue: https://github.com/apache/drill/pull/805 This my bad habit to regenerate metadata left from the times before fix DRILL-5660. These functional test failures are the result of the mistake that I have made when resolving merge conflicts

[GitHub] drill pull request #957: DRILL-5809 Fixed Option Storage Backward Compatibil...

2017-09-25 Thread ilooner
Github user ilooner commented on a diff in the pull request: https://github.com/apache/drill/pull/957#discussion_r140946604 --- Diff: common/src/test/java/org/apache/drill/testutils/DirTestWatcher.java --- @@ -0,0 +1,61 @@ +/** + * Licensed to the Apache Software

[GitHub] drill pull request #905: DRILL-1162: Fix OOM for hash join operator when the...

2017-09-25 Thread vvysotskyi
Github user vvysotskyi commented on a diff in the pull request: https://github.com/apache/drill/pull/905#discussion_r140730762 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/planner/cost/DrillRelMdMaxRowCount.java --- @@ -0,0 +1,71 @@ +/* +* Licensed to the

[GitHub] drill issue #805: Drill-4139: Exception while trying to prune partition. jav...

2017-09-25 Thread vvysotskyi
Github user vvysotskyi commented on the issue: https://github.com/apache/drill/pull/805 @paul-rogers, test data should be regenerated (using the Drill version with this fix) before running functional tests since these tests use metadata cache files. Regarding unit test failures:

[GitHub] drill issue #905: DRILL-1162: Fix OOM for hash join operator when the right ...

2017-09-25 Thread vvysotskyi
Github user vvysotskyi commented on the issue: https://github.com/apache/drill/pull/905 Since Drill does not have enough information about tables in the planning time to avoid this OOM and since without using the `getMaxRowCount()` for `ScanPrel`, current approach could not be used

[GitHub] drill pull request #889: DRILL-5691: enhance scalar sub queries checking for...

2017-09-25 Thread weijietong
Github user weijietong commented on a diff in the pull request: https://github.com/apache/drill/pull/889#discussion_r140756869 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/planner/cost/DrillRelMdMaxRowCount.java --- @@ -0,0 +1,42 @@ +/* + * Licensed to the

[GitHub] drill issue #905: DRILL-1162: Fix OOM for hash join operator when the right ...

2017-09-25 Thread vvysotskyi
Github user vvysotskyi commented on the issue: https://github.com/apache/drill/pull/905 @jinfengni it was an inflated example, but considering the case of multiple joins and when tables have several repeated values, the result will be the same. ---

[GitHub] drill pull request #905: DRILL-1162: Fix OOM for hash join operator when the...

2017-09-25 Thread vvysotskyi
Github user vvysotskyi closed the pull request at: https://github.com/apache/drill/pull/905 ---