[GitHub] drill pull request #1228: DRILL-6307: Handle empty batches in record batch s...

2018-04-26 Thread ppadma
Github user ppadma commented on a diff in the pull request: https://github.com/apache/drill/pull/1228#discussion_r184483395 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/record/RecordBatchSizer.java --- @@ -536,6 +556,11 @@ public ColumnSize getColumn(String name)

Re: Display column data type without code

2018-04-26 Thread Paul Rogers
Thanks! A it turns out, typeof() is not documented on the Apache Drill website which is why I was confused. Maybe we should document it. Just tried typeof(). Turns out it only returns the type, not the nullability or repeated mode. Is there a separate function to get the "mode" information? -

[GitHub] drill issue #1214: DRILL-6331: Revisit Hive Drill native parquet implementat...

2018-04-26 Thread parthchandra
Github user parthchandra commented on the issue: https://github.com/apache/drill/pull/1214 Looks Good. Thanks for making the changes Arina. +1 ---

[GitHub] drill issue #1184: DRILL-6242 - Use java.sql.[Date|Time|Timestamp] classes t...

2018-04-26 Thread parthchandra
Github user parthchandra commented on the issue: https://github.com/apache/drill/pull/1184 > But, if April 15 is your birthday, it is your birthday in all timezones. We don't say your birthday (or order date, or newspaper issue date or...) is one day in, say London and another day

[GitHub] drill pull request #1225: DRILL-6272: Refactor dynamic UDFs and function ini...

2018-04-26 Thread vrozov
Github user vrozov commented on a diff in the pull request: https://github.com/apache/drill/pull/1225#discussion_r184245358 --- Diff: pom.xml --- @@ -798,7 +798,7 @@ com.googlecode.jmockit jmockit - 1.3 + 1.7 --- End diff --

[GitHub] drill pull request #1225: DRILL-6272: Refactor dynamic UDFs and function ini...

2018-04-26 Thread vrozov
Github user vrozov commented on a diff in the pull request: https://github.com/apache/drill/pull/1225#discussion_r184411700 --- Diff: exec/java-exec/pom.xml --- @@ -593,6 +593,48 @@ netty-tcnative ${netty.tcnative.classifier} + +

[GitHub] drill issue #1232: DRILL-6094: Decimal data type enhancements

2018-04-26 Thread parthchandra
Github user parthchandra commented on the issue: https://github.com/apache/drill/pull/1232 +1 for the C++ part. Looks really good. ---

[GitHub] drill issue #1184: DRILL-6242 - Use java.sql.[Date|Time|Timestamp] classes t...

2018-04-26 Thread jiang-wu
Github user jiang-wu commented on the issue: https://github.com/apache/drill/pull/1184 Yes. There are at least two issues. One is about how Drill represent Date, Time, Timestamp internally using a UTC based instant representation and fudges the timezone in order to make

[GitHub] drill issue #1238: DRILL-6281: Refactor TimedRunnable

2018-04-26 Thread vrozov
Github user vrozov commented on the issue: https://github.com/apache/drill/pull/1238 The step is necessary to do RCA for DRILL-5908. There are way too many issues with the current implementation to list them in JIRA or PR and the major issue is the usage of homegrown solutions where

[GitHub] drill issue #1238: DRILL-6281: Refactor TimedRunnable

2018-04-26 Thread vrozov
Github user vrozov commented on the issue: https://github.com/apache/drill/pull/1238 I did not change how tasks (`Runnable` or `Callable`) behave and did not look into converting `Callable/Runnable` to a `ForkJoinTask`. Whether existing tasks can be scheduled recursively or not

Re: Display column data type without code

2018-04-26 Thread Rob Wu
Hi Paul, You could also use DESCRIBE (https://drill.apache.org/docs/describe/). 0: jdbc:drill:drillbit=localhost:31010> describe `hive.default`.`integer_table` . . . . . . . . . . . . . . . . . . . > ; +--++--+ | COLUMN_NAME | DATA_TYPE |

[GitHub] drill pull request #1184: DRILL-6242 - Use java.sql.[Date|Time|Timestamp] cl...

2018-04-26 Thread jiang-wu
Github user jiang-wu commented on a diff in the pull request: https://github.com/apache/drill/pull/1184#discussion_r184424638 --- Diff: exec/vector/src/main/codegen/templates/FixedValueVectors.java --- @@ -509,15 +509,15 @@ public long getTwoAsLong(int index) { public

[GitHub] drill issue #1214: DRILL-6331: Revisit Hive Drill native parquet implementat...

2018-04-26 Thread arina-ielchiieva
Github user arina-ielchiieva commented on the issue: https://github.com/apache/drill/pull/1214 @parthchandra added two new commits: 1. reverted custom Stopwatch implementation and used logger checks instead. 2. allowed to create several non-tracking fs but only one tracking per

[GitHub] drill issue #1238: DRILL-6281: Refactor TimedRunnable

2018-04-26 Thread vrozov
Github user vrozov commented on the issue: https://github.com/apache/drill/pull/1238 There is not enough info available to debug and/or troubleshoot DRILL-5908 and I prefer instead of trying to find bugs in homegrown solution replace it with Java out of the box functionality and at

[GitHub] drill issue #1238: DRILL-6281: Refactor TimedRunnable

2018-04-26 Thread parthchandra
Github user parthchandra commented on the issue: https://github.com/apache/drill/pull/1238 Fair enough. But that *still* does not give me a clue about the problem(s) you were trying to fix, or how the refactoring helps. Is the cause of the problem in TimedRunnable? Not an

[jira] [Resolved] (DRILL-4156) Parquet group converter cannot cope with ENUM original type values

2018-04-26 Thread Arina Ielchiieva (JIRA)
[ https://issues.apache.org/jira/browse/DRILL-4156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arina Ielchiieva resolved DRILL-4156. - Resolution: Fixed Fixed in DRILL-5971. > Parquet group converter cannot cope with ENUM

[GitHub] drill issue #1184: DRILL-6242 - Use java.sql.[Date|Time|Timestamp] classes t...

2018-04-26 Thread paul-rogers
Github user paul-rogers commented on the issue: https://github.com/apache/drill/pull/1184 One additional note. We noted that JDBC does not support the idea of a nested tuple (a Drill "map".) JDBC does support columns that return a Java object. To bridge the gap, Drill returns a Map

[GitHub] drill issue #1184: DRILL-6242 - Use java.sql.[Date|Time|Timestamp] classes t...

2018-04-26 Thread paul-rogers
Github user paul-rogers commented on the issue: https://github.com/apache/drill/pull/1184 @parthchandra, the point about the birthday is that is is one of those dates that is implied relative to where you are. You celebrate it the same day regardless of where you are in the world.

[GitHub] drill pull request #1234: DRILL-5927: Fixed memory leak in TestBsonRecordRea...

2018-04-26 Thread parthchandra
Github user parthchandra commented on a diff in the pull request: https://github.com/apache/drill/pull/1234#discussion_r184549758 --- Diff: exec/java-exec/src/test/java/org/apache/drill/exec/store/bson/TestBsonRecordReader.java --- @@ -45,21 +47,24 @@ import

[GitHub] drill issue #1144: DRILL-6202: Deprecate usage of IndexOutOfBoundsException ...

2018-04-26 Thread parthchandra
Github user parthchandra commented on the issue: https://github.com/apache/drill/pull/1144 Looks good. +1 ---

[GitHub] drill pull request #1228: DRILL-6307: Handle empty batches in record batch s...

2018-04-26 Thread paul-rogers
Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/1228#discussion_r184590500 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/record/RecordBatchSizer.java --- @@ -536,6 +556,11 @@ public ColumnSize getColumn(String

[GitHub] drill pull request #1237: DRILL-6348: Fixed code so that Unordered Receiver ...

2018-04-26 Thread vrozov
Github user vrozov commented on a diff in the pull request: https://github.com/apache/drill/pull/1237#discussion_r184590436 --- Diff: exec/memory/base/src/main/java/org/apache/drill/exec/memory/AllocationManager.java --- @@ -253,10 +261,12 @@ public boolean transferBalance(final

[jira] [Created] (DRILL-6360) Document the typeof() function

2018-04-26 Thread Paul Rogers (JIRA)
Paul Rogers created DRILL-6360: -- Summary: Document the typeof() function Key: DRILL-6360 URL: https://issues.apache.org/jira/browse/DRILL-6360 Project: Apache Drill Issue Type: Improvement

[jira] [Created] (DRILL-6361) Provide a dataTypeOf() or modeOf() function

2018-04-26 Thread Paul Rogers (JIRA)
Paul Rogers created DRILL-6361: -- Summary: Provide a dataTypeOf() or modeOf() function Key: DRILL-6361 URL: https://issues.apache.org/jira/browse/DRILL-6361 Project: Apache Drill Issue Type:

[jira] [Created] (DRILL-6362) typeof() lies about types

2018-04-26 Thread Paul Rogers (JIRA)
Paul Rogers created DRILL-6362: -- Summary: typeof() lies about types Key: DRILL-6362 URL: https://issues.apache.org/jira/browse/DRILL-6362 Project: Apache Drill Issue Type: Improvement

[GitHub] drill pull request #1237: DRILL-6348: Fixed code so that Unordered Receiver ...

2018-04-26 Thread vrozov
Github user vrozov commented on a diff in the pull request: https://github.com/apache/drill/pull/1237#discussion_r184596895 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/record/RawFragmentBatch.java --- @@ -77,4 +83,46 @@ public long getByteCount() { public

[GitHub] drill pull request #1237: DRILL-6348: Fixed code so that Unordered Receiver ...

2018-04-26 Thread sachouche
Github user sachouche commented on a diff in the pull request: https://github.com/apache/drill/pull/1237#discussion_r184572864 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/unorderedreceiver/UnorderedReceiverBatch.java --- @@ -149,25 +149,32 @@

[GitHub] drill pull request #1237: DRILL-6348: Fixed code so that Unordered Receiver ...

2018-04-26 Thread sachouche
Github user sachouche commented on a diff in the pull request: https://github.com/apache/drill/pull/1237#discussion_r184574405 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/unorderedreceiver/UnorderedReceiverBatch.java --- @@ -149,25 +149,32 @@

[GitHub] drill pull request #1237: DRILL-6348: Fixed code so that Unordered Receiver ...

2018-04-26 Thread sachouche
Github user sachouche commented on a diff in the pull request: https://github.com/apache/drill/pull/1237#discussion_r184573646 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/unorderedreceiver/UnorderedReceiverBatch.java --- @@ -149,25 +149,32 @@

[GitHub] drill pull request #1237: DRILL-6348: Fixed code so that Unordered Receiver ...

2018-04-26 Thread sachouche
Github user sachouche commented on a diff in the pull request: https://github.com/apache/drill/pull/1237#discussion_r184575686 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/unorderedreceiver/UnorderedReceiverBatch.java --- @@ -149,25 +149,32 @@

[GitHub] drill pull request #1237: DRILL-6348: Fixed code so that Unordered Receiver ...

2018-04-26 Thread sachouche
Github user sachouche commented on a diff in the pull request: https://github.com/apache/drill/pull/1237#discussion_r184574622 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/unorderedreceiver/UnorderedReceiverBatch.java --- @@ -201,6 +208,11 @@ public

[GitHub] drill issue #1237: DRILL-6348: Fixed code so that Unordered Receiver reports...

2018-04-26 Thread sachouche
Github user sachouche commented on the issue: https://github.com/apache/drill/pull/1237 @vrozov, your observation is valid, we need more JIRAs to fix the reporting problem **Current Fix** - At this time, the UnorderedReceiver didn't account for any consumed memory - This

[GitHub] drill pull request #1237: DRILL-6348: Fixed code so that Unordered Receiver ...

2018-04-26 Thread vrozov
Github user vrozov commented on a diff in the pull request: https://github.com/apache/drill/pull/1237#discussion_r184585775 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/unorderedreceiver/UnorderedReceiverBatch.java --- @@ -149,25 +149,32 @@ private

[GitHub] drill pull request #1237: DRILL-6348: Fixed code so that Unordered Receiver ...

2018-04-26 Thread vrozov
Github user vrozov commented on a diff in the pull request: https://github.com/apache/drill/pull/1237#discussion_r184585959 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/unorderedreceiver/UnorderedReceiverBatch.java --- @@ -149,25 +149,32 @@ private

[GitHub] drill pull request #1237: DRILL-6348: Fixed code so that Unordered Receiver ...

2018-04-26 Thread vrozov
Github user vrozov commented on a diff in the pull request: https://github.com/apache/drill/pull/1237#discussion_r184586222 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/unorderedreceiver/UnorderedReceiverBatch.java --- @@ -149,25 +149,32 @@ private

[GitHub] drill issue #1237: DRILL-6348: Fixed code so that Unordered Receiver reports...

2018-04-26 Thread vrozov
Github user vrozov commented on the issue: https://github.com/apache/drill/pull/1237 IMO, it is better not to report memory usage at all compared to reporting a wrong number. In case incoming batches are accumulated in a queue, they should be reported as owned by a receiver. Taking

[GitHub] drill pull request #1237: DRILL-6348: Fixed code so that Unordered Receiver ...

2018-04-26 Thread vrozov
Github user vrozov commented on a diff in the pull request: https://github.com/apache/drill/pull/1237#discussion_r184589826 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/unorderedreceiver/UnorderedReceiverBatch.java --- @@ -201,6 +208,11 @@ public

[GitHub] drill issue #1184: DRILL-6242 - Use java.sql.[Date|Time|Timestamp] classes t...

2018-04-26 Thread parthchandra
Github user parthchandra commented on the issue: https://github.com/apache/drill/pull/1184 Putting aside the discussion on date/time/timezone for the moment, @jiang-wu let's say getObject returns to you an object that implements java.sql.{Struct|Array}. You now use the Struct|Array

[GitHub] drill pull request #1237: DRILL-6348: Fixed code so that Unordered Receiver ...

2018-04-26 Thread vrozov
Github user vrozov commented on a diff in the pull request: https://github.com/apache/drill/pull/1237#discussion_r184554299 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/unorderedreceiver/UnorderedReceiverBatch.java --- @@ -149,25 +149,32 @@ private

[GitHub] drill pull request #1237: DRILL-6348: Fixed code so that Unordered Receiver ...

2018-04-26 Thread vrozov
Github user vrozov commented on a diff in the pull request: https://github.com/apache/drill/pull/1237#discussion_r184558425 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/unorderedreceiver/UnorderedReceiverBatch.java --- @@ -149,25 +149,32 @@ private

[GitHub] drill pull request #1237: DRILL-6348: Fixed code so that Unordered Receiver ...

2018-04-26 Thread vrozov
Github user vrozov commented on a diff in the pull request: https://github.com/apache/drill/pull/1237#discussion_r184554436 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/unorderedreceiver/UnorderedReceiverBatch.java --- @@ -149,25 +149,32 @@ private

[GitHub] drill pull request #1237: DRILL-6348: Fixed code so that Unordered Receiver ...

2018-04-26 Thread vrozov
Github user vrozov commented on a diff in the pull request: https://github.com/apache/drill/pull/1237#discussion_r184559429 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/unorderedreceiver/UnorderedReceiverBatch.java --- @@ -201,6 +208,11 @@ public

[GitHub] drill issue #1184: DRILL-6242 - Use java.sql.[Date|Time|Timestamp] classes t...

2018-04-26 Thread jiang-wu
Github user jiang-wu commented on the issue: https://github.com/apache/drill/pull/1184 @parthchandra Just to clarify on the JDBC comment. What do you mean by "Json representation"? Do you instead mean the "Local[Date|Time]" class representation? There are no "Json" being returned

[GitHub] drill pull request #1214: DRILL-6331: Revisit Hive Drill native parquet impl...

2018-04-26 Thread arina-ielchiieva
Github user arina-ielchiieva commented on a diff in the pull request: https://github.com/apache/drill/pull/1214#discussion_r184401600 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/ops/BaseOperatorContext.java --- @@ -158,25 +159,26 @@ public void close() {

[GitHub] drill issue #1184: DRILL-6242 - Use java.sql.[Date|Time|Timestamp] classes t...

2018-04-26 Thread jiang-wu
Github user jiang-wu commented on the issue: https://github.com/apache/drill/pull/1184 Actually, JDBC representation is not he hard problem here. I ran into most of the problems dealing with the timezones surrounding the data|time|timestamp. java.sql.Struct and Array are interfaces

[GitHub] drill issue #1227: DRILL-6236: batch sizing for hash join

2018-04-26 Thread ilooner
Github user ilooner commented on the issue: https://github.com/apache/drill/pull/1227 @ppadma Please fix travis failure ---

[GitHub] drill issue #1232: DRILL-6094: Decimal data type enhancements

2018-04-26 Thread kkhatua
Github user kkhatua commented on the issue: https://github.com/apache/drill/pull/1232 I would recommend we wait till we have all testing complete, since this is a big feature. ---