[jira] [Commented] (DRILL-7393) Revisit Drill tests to ensure that patching is executed before any test run

2019-12-03 Thread ASF GitHub Bot (Jira) 


[ 
https://issues.apache.org/jira/browse/DRILL-7393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16987457#comment-16987457
 ] 

ASF GitHub Bot commented on DRILL-7393:
---

paul-rogers commented on issue #1910: DRILL-7393: Revisit Drill tests to ensure 
that patching is executed b…
URL: https://github.com/apache/drill/pull/1910#issuecomment-561449338
 
 
   @vvysotskyi, you got me on that one. If this is a release blocker, then, 
yes, we have to live with the best fix we can do right now. 
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Revisit Drill tests to ensure that patching is executed before any test run
> ---
>
> Key: DRILL-7393
> URL: https://issues.apache.org/jira/browse/DRILL-7393
> Project: Apache Drill
>  Issue Type: Task
>Affects Versions: 1.16.0, 1.17.0
>Reporter: Arina Ielchiieva
>Assignee: Anton Gozhiy
>Priority: Major
>  Labels: ready-to-commit
>
> Apache Drill patches some Protobuf and Guava classes (see GuavaPatcher, 
> ProtobufPatcher), patching should be done before classes to be patched are 
> loaded. That's why this operation is executed in static block in Drillbit 
> class.
> Some tests in java-exec module use Drillbit class, some extend DrillTest 
> class, both of them patch Guava. But there are some tests that do not call 
> patcher but load classes to be patched. For example, 
> {{org.apache.drill.exec.sql.TestSqlBracketlessSyntax}} loads Guava 
> Preconditions class. If such tests run before tests that require patching, 
> tests run will fail since patching won't be successful. Patchers code does 
> not fail application if patching was not complete, just logs warning 
> ({{logger.warn("Unable to patch Guava classes.", e);}}), so sometimes it hard 
> to identify unit tests failure root cause.
> We need to revisit all Drill tests to ensure that all of them extend common 
> test base class which patchers Protobuf and Guava classes in static block. 
> Also refactor Patcher classes to have assert to fail if patching fails during 
> unit testing if there are any problems.
> After all tests are revised, we can remove {{metastore-test}} execution from 
> main.xml in {{maven-surefire-plugin}} which was added to ensure that all 
> Metastore tests run in a separate JVM where patching is done in first place 
> since Iceberg Metastore heavily depends on patched Guava Preconditions class.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (DRILL-7359) Add support for DICT type in RowSet Framework

2019-12-03 Thread ASF GitHub Bot (Jira) 


[ 
https://issues.apache.org/jira/browse/DRILL-7359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16987453#comment-16987453
 ] 

ASF GitHub Bot commented on DRILL-7359:
---

paul-rogers commented on issue #1870: DRILL-7359: Add support for DICT type in 
RowSet Framework
URL: https://github.com/apache/drill/pull/1870#issuecomment-561448583
 
 
   On `ProjectionType`, if the `SchemaPath` is, say, `foo`, then the 
`ProjectionType` will be `SCALAR`. I can see that this name is misleading. 
Perhaps `SIMPLE` is better. We don't know that the column **is** scalar, we 
just know that it is consistent with a scalar. It could be a map, array or dict 
as well.
   
   But, if you do add a `SchemaPath` representation for `foo['bar']`, then, 
yes, that can only be a `DICT`, so we can introduce a `DICT` projection type.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Add support for DICT type in RowSet Framework
> -
>
> Key: DRILL-7359
> URL: https://issues.apache.org/jira/browse/DRILL-7359
> Project: Apache Drill
>  Issue Type: New Feature
>Reporter: Bohdan Kazydub
>Assignee: Bohdan Kazydub
>Priority: Major
> Fix For: 1.17.0
>
>
> Add support for new DICT data type (see DRILL-7096) in RowSet Framework



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (DRILL-7359) Add support for DICT type in RowSet Framework

2019-12-03 Thread ASF GitHub Bot (Jira) 


[ 
https://issues.apache.org/jira/browse/DRILL-7359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16987446#comment-16987446
 ] 

ASF GitHub Bot commented on DRILL-7359:
---

paul-rogers commented on pull request #1870: DRILL-7359: Add support for DICT 
type in RowSet Framework
URL: https://github.com/apache/drill/pull/1870#discussion_r353522274
 
 

 ##
 File path: 
exec/vector/src/main/java/org/apache/drill/exec/vector/accessor/reader/UnionReaderImpl.java
 ##
 @@ -268,4 +274,51 @@ public String getAsString() {
 }
 return requireReader(type).getAsString();
   }
+
+  private UnionReaderImpl getNullReader() {
+AbstractObjectReader[] nullVariants = new 
AbstractObjectReader[variants.length];
+for (int i = 0; i < variants.length; i++) {
+  nullVariants[i] = variants[i].createNullReader();
+}
+return new NullUnionReader(schema(), unionAccessor, nullVariants);
+  }
+
+  private static class NullUnionReader extends UnionReaderImpl {
+
+private NullUnionReader(ColumnMetadata metadata, VectorAccessor va, 
AbstractObjectReader[] variants) {
 
 Review comment:
   @KazydubB, thanks for the explanation. It clarifies something I wondered 
about: the role of the null state readers. Let's talk a bit about the intended 
semantics for the readers.
   
   I should be able to write code that caches the readers:
   
   ```
   DictReader dr = rowReader.dict("myDict");
   ScalarWriter kw = dr.key().scalar();
   MapWriter vr = dr.value.tuple();
   ```
   
   That is, I should be able to use the same key reader (`kr`) and value reader 
(`vr`) for all members. Then, I should either be able to iterate over the 
values:
   
   ```
   while (dr.next()) {
 System.out.println(kr.getString());
 System.out.println(vr.getAsString());
   }
   ```
   
   If things work that way, then we don't need a special reader for the null 
state: the value reader (here, `vr`) would simply return `true` from `isNull()`:
   
   ```
   while (dr.next()) {
 System.out.println(kr.getString());
 if (vr.isNull()) {
   System.out.println("NULL");
 } else {
   System.out.println(vr.getAsString());
 }
   }
   ```
   
   (Ignoring, for the moment, that `getAsString()` already does a null check.)
   
   Note that, if we read past the end, both the key and value writers will 
point to nothing. I think that, for the array reader, accessing the value one 
past the end is an error. For `DICT`, we could simply declare that the value is 
NULL.
   
   Then, I was going to sketch out the same behavior if we search for a key and 
found...nothing. Seems that the `DictReader` has no way to search for a key. If 
it did, it would not be clear what type to use for the key. So, that is a bit 
of a mess.
   
   Let's assume we did have something to find a specific value, like we do for 
the arrays where we have `setPosn(int index)`. Maybe we have `findStringKey()` 
etc. For testing, we could just have `findKey(Object key)` and do a switch on 
type as we do for the other readers. So:
   
   ```
 dr.findStringKey("foo");
 if (vr.isNull()) {
   System.out.println("NULL");
 } else {
   System.out.println(vr.getAsString());
 }
   ```
   
   Or, even:
   
   ```
 if (dr.findStringKey("foo")) {
   System.out.println(vr.getAsString());
 }
   ```
   
   The only way to find the value is to do a linear search. If we do a linear 
search, and no value exists, the index will point one past the end of the dict. 
And, above, we suggested that the key and value readers could return `true` for 
`isNull()` in those cases.
   
   Given this, unless I'm missing something, I don't see that we need any kind 
of null value reader at all. In fact, I'd even say that we don't want such a 
reader if we'd have to do:
   
   ```
   ObjectReader kr = dr.findStringKey("foo");
   if (kr.isNull) {
 // kr is a special null reader
   } else {
 // kr is a Map reader
   }
   ```
   
   The above clutters the code and makes it impossible to cache the value 
reader, which will slow inner-most loops that want to use the reader 
abstractions.
   
   What do you think?
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Add support for DICT type in RowSet Framework
> -
>
> Key: DRILL-7359
> URL: https://issues.apache.org/jira/browse/DRILL-7359
> Project: Apache Drill
>  Issue Type: New Feature
>Reporter: Bohdan Kazydub
>Assignee: Bohdan Kazydub
>Priority: Major
> Fix For: 1.17.0
>
>
> Add support for new DICT data type (see

[jira] [Created] (DRILL-7462) Fix Links and Typos in Documentation

2019-12-03 Thread Charles Givre (Jira) 
Charles Givre created DRILL-7462:


 Summary: Fix Links and Typos in Documentation 
 Key: DRILL-7462
 URL: https://issues.apache.org/jira/browse/DRILL-7462
 Project: Apache Drill
  Issue Type: Bug
  Components: Documentation
Affects Versions: 1.16.0
Reporter: Charles Givre
Assignee: Charles Givre
 Fix For: 1.17.0


This issue fixes several minor typos in the documentation:

* Fix en-dash code points in command line [1]
* Use SSL for download link[2] 
* Update Slack link in README [3]

[1] https://github.com/apache/drill/pull/1909
[2] https://github.com/apache/drill/pull/1908 
[3] 
https://join.slack.com/t/apache-drill/shared_invite/enQtNTQ4MjM1MDA3MzQ2LTJlYmUxMTRkMmUwYmQ2NTllYmFmMjU4MDk0NjYwZjBmYjg0MDZmOTE2ZDg0ZjBlYmI3Yjc4Y2I2NTQyNGVlZTc




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (DRILL-7460) Exclude classes which cannot be loaded from JDBC Driver

2019-12-03 Thread Vova Vysotskyi (Jira) 


[ 
https://issues.apache.org/jira/browse/DRILL-7460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16987106#comment-16987106
 ] 

Vova Vysotskyi commented on DRILL-7460:
---

Except excluding these classes from the Jar, it would be handfull to add a test 
that checks that all "transitive" classes can be loaded (including classes used 
in the methods) to avoid issues in runtime. This test can be implemented using 
ASM visitors. The preliminary check showed that there are more than 400 classes 
(including transitive ones) that cannot be loaded.

> Exclude classes which cannot be loaded from JDBC Driver 
> 
>
> Key: DRILL-7460
> URL: https://issues.apache.org/jira/browse/DRILL-7460
> Project: Apache Drill
>  Issue Type: Bug
>Affects Versions: 1.16.0
>Reporter: Vova Vysotskyi
>Assignee: Vova Vysotskyi
>Priority: Minor
> Fix For: 1.17.0
>
>
> When using drill-jdbc-driver with SQuirreLSQL, it logs the following errors:
> {noformat}
> 2019-11-28 15:22:53,406 [Thread-0] DEBUG 
> net.sourceforge.squirrel_sql.client.gui.db.DriverInternalFrame  - 
> getFileFromUrl: 
> fileUri.getPath()=/Users/vova/work/drill/distribution/target/apache-drill-1.17.0-SNAPSHOT/apache-drill-1.17.0-SNAPSHOT/jars/jdbc-driver/drill-jdbc-all-1.17.0-SNAPSHOT.jar
> 2019-11-28 15:22:53,431 [Thread-0] INFO  
> net.sourceforge.squirrel_sql.client.gui.db.DriverInternalFrame  - Failed to 
> load oadd.org.apache.drill.common.util.RepeatTestRule in 
> /Users/vova/work/drill/distribution/target/apache-drill-1.17.0-SNAPSHOT/apache-drill-1.17.0-SNAPSHOT/jars/jdbc-driver/drill-jdbc-all-1.17.0-SNAPSHOT.jar
>  to check if it is assignable to java.sql.Driver. Reason: 
> java.lang.NoClassDefFoundError: org/junit/rules/TestRule
> 2019-11-28 15:22:53,432 [Thread-0] INFO  
> net.sourceforge.squirrel_sql.client.gui.db.DriverInternalFrame  - Failed to 
> load oadd.org.apache.drill.common.util.RepeatTestRule$RepeatStatement in 
> /Users/vova/work/drill/distribution/target/apache-drill-1.17.0-SNAPSHOT/apache-drill-1.17.0-SNAPSHOT/jars/jdbc-driver/drill-jdbc-all-1.17.0-SNAPSHOT.jar
>  to check if it is assignable to java.sql.Driver. Reason: 
> java.lang.NoClassDefFoundError: org/junit/runners/model/Statement
> 2019-11-28 15:22:53,458 [Thread-0] INFO  
> net.sourceforge.squirrel_sql.client.gui.db.DriverInternalFrame  - Failed to 
> load 
> oadd.org.apache.drill.exec.proto.SchemaGeneralRPCProtos$CompleteRpcMessage$BuilderSchema
>  in 
> /Users/vova/work/drill/distribution/target/apache-drill-1.17.0-SNAPSHOT/apache-drill-1.17.0-SNAPSHOT/jars/jdbc-driver/drill-jdbc-all-1.17.0-SNAPSHOT.jar
>  to check if it is assignable to java.sql.Driver. Reason: 
> java.lang.NoClassDefFoundError: oadd/io/protostuff/Schema
> 2019-11-28 15:22:53,458 [Thread-0] INFO  
> net.sourceforge.squirrel_sql.client.gui.db.DriverInternalFrame  - Failed to 
> load 
> oadd.org.apache.drill.exec.proto.SchemaBitControl$BitControlHandshake$BuilderSchema
>  in 
> /Users/vova/work/drill/distribution/target/apache-drill-1.17.0-SNAPSHOT/apache-drill-1.17.0-SNAPSHOT/jars/jdbc-driver/drill-jdbc-all-1.17.0-SNAPSHOT.jar
>  to check if it is assignable to java.sql.Driver. Reason: 
> java.lang.NoClassDefFoundError: oadd/io/protostuff/Schema
> 2019-11-28 15:22:53,468 [Thread-0] INFO  
> net.sourceforge.squirrel_sql.client.gui.db.DriverInternalFrame  - Failed to 
> load 
> oadd.org.apache.drill.exec.proto.SchemaUserBitShared$ExceptionWrapper$MessageSchema
>  in 
> /Users/vova/work/drill/distribution/target/apache-drill-1.17.0-SNAPSHOT/apache-drill-1.17.0-SNAPSHOT/jars/jdbc-driver/drill-jdbc-all-1.17.0-SNAPSHOT.jar
>  to check if it is assignable to java.sql.Driver. Reason: 
> java.lang.NoClassDefFoundError: oadd/io/protostuff/Schema
> 2019-11-28 15:22:53,468 [Thread-0] INFO  
> net.sourceforge.squirrel_sql.client.gui.db.DriverInternalFrame  - Failed to 
> load 
> oadd.org.apache.drill.exec.proto.SchemaUserProtos$CreatePreparedStatementReq$MessageSchema
>  in 
> /Users/vova/work/drill/distribution/target/apache-drill-1.17.0-SNAPSHOT/apache-drill-1.17.0-SNAPSHOT/jars/jdbc-driver/drill-jdbc-all-1.17.0-SNAPSHOT.jar
>  to check if it is assignable to java.sql.Driver. Reason: 
> java.lang.NoClassDefFoundError: oadd/io/protostuff/Schema
> 2019-11-28 15:22:53,470 [Thread-0] INFO  
> net.sourceforge.squirrel_sql.client.gui.db.DriverInternalFrame  - Failed to 
> load 
> oadd.org.apache.drill.exec.proto.SchemaUserBitShared$MetricValue$MessageSchema
>  in 
> /Users/vova/work/drill/distribution/target/apache-drill-1.17.0-SNAPSHOT/apache-drill-1.17.0-SNAPSHOT/jars/jdbc-driver/drill-jdbc-all-1.17.0-SNAPSHOT.jar
>  to check if it is assignable to java.sql.Driver. Reason: 
> java.lang.NoClassDefFoundError: oadd/io/protostuff/Schema
> 2019-11-28 15:22:53,471 [Thread-0] INFO  
>

[jira] [Commented] (DRILL-7359) Add support for DICT type in RowSet Framework

2019-12-03 Thread ASF GitHub Bot (Jira) 


[ 
https://issues.apache.org/jira/browse/DRILL-7359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986826#comment-16986826
 ] 

ASF GitHub Bot commented on DRILL-7359:
---

KazydubB commented on issue #1870: DRILL-7359: Add support for DICT type in 
RowSet Framework
URL: https://github.com/apache/drill/pull/1870#issuecomment-561135023
 
 
   @paul-rogers, `ProjectionType` now has no notion of `MinorType#DICT` and 
treats it as `ProjectionType#SCALAR`. And yes, we do need to allow Python-like 
syntax for the type. (My bad, haven't made changes to this).
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Add support for DICT type in RowSet Framework
> -
>
> Key: DRILL-7359
> URL: https://issues.apache.org/jira/browse/DRILL-7359
> Project: Apache Drill
>  Issue Type: New Feature
>Reporter: Bohdan Kazydub
>Assignee: Bohdan Kazydub
>Priority: Major
> Fix For: 1.17.0
>
>
> Add support for new DICT data type (see DRILL-7096) in RowSet Framework



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Assigned] (DRILL-7460) Exclude classes which cannot be loaded from JDBC Driver

2019-12-03 Thread Vova Vysotskyi (Jira) 


 [ 
https://issues.apache.org/jira/browse/DRILL-7460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vova Vysotskyi reassigned DRILL-7460:
-

Assignee: Vova Vysotskyi

> Exclude classes which cannot be loaded from JDBC Driver 
> 
>
> Key: DRILL-7460
> URL: https://issues.apache.org/jira/browse/DRILL-7460
> Project: Apache Drill
>  Issue Type: Bug
>Affects Versions: 1.16.0
>Reporter: Vova Vysotskyi
>Assignee: Vova Vysotskyi
>Priority: Minor
>
> When using drill-jdbc-driver with SQuirreLSQL, it logs the following errors:
> {noformat}
> 2019-11-28 15:22:53,406 [Thread-0] DEBUG 
> net.sourceforge.squirrel_sql.client.gui.db.DriverInternalFrame  - 
> getFileFromUrl: 
> fileUri.getPath()=/Users/vova/work/drill/distribution/target/apache-drill-1.17.0-SNAPSHOT/apache-drill-1.17.0-SNAPSHOT/jars/jdbc-driver/drill-jdbc-all-1.17.0-SNAPSHOT.jar
> 2019-11-28 15:22:53,431 [Thread-0] INFO  
> net.sourceforge.squirrel_sql.client.gui.db.DriverInternalFrame  - Failed to 
> load oadd.org.apache.drill.common.util.RepeatTestRule in 
> /Users/vova/work/drill/distribution/target/apache-drill-1.17.0-SNAPSHOT/apache-drill-1.17.0-SNAPSHOT/jars/jdbc-driver/drill-jdbc-all-1.17.0-SNAPSHOT.jar
>  to check if it is assignable to java.sql.Driver. Reason: 
> java.lang.NoClassDefFoundError: org/junit/rules/TestRule
> 2019-11-28 15:22:53,432 [Thread-0] INFO  
> net.sourceforge.squirrel_sql.client.gui.db.DriverInternalFrame  - Failed to 
> load oadd.org.apache.drill.common.util.RepeatTestRule$RepeatStatement in 
> /Users/vova/work/drill/distribution/target/apache-drill-1.17.0-SNAPSHOT/apache-drill-1.17.0-SNAPSHOT/jars/jdbc-driver/drill-jdbc-all-1.17.0-SNAPSHOT.jar
>  to check if it is assignable to java.sql.Driver. Reason: 
> java.lang.NoClassDefFoundError: org/junit/runners/model/Statement
> 2019-11-28 15:22:53,458 [Thread-0] INFO  
> net.sourceforge.squirrel_sql.client.gui.db.DriverInternalFrame  - Failed to 
> load 
> oadd.org.apache.drill.exec.proto.SchemaGeneralRPCProtos$CompleteRpcMessage$BuilderSchema
>  in 
> /Users/vova/work/drill/distribution/target/apache-drill-1.17.0-SNAPSHOT/apache-drill-1.17.0-SNAPSHOT/jars/jdbc-driver/drill-jdbc-all-1.17.0-SNAPSHOT.jar
>  to check if it is assignable to java.sql.Driver. Reason: 
> java.lang.NoClassDefFoundError: oadd/io/protostuff/Schema
> 2019-11-28 15:22:53,458 [Thread-0] INFO  
> net.sourceforge.squirrel_sql.client.gui.db.DriverInternalFrame  - Failed to 
> load 
> oadd.org.apache.drill.exec.proto.SchemaBitControl$BitControlHandshake$BuilderSchema
>  in 
> /Users/vova/work/drill/distribution/target/apache-drill-1.17.0-SNAPSHOT/apache-drill-1.17.0-SNAPSHOT/jars/jdbc-driver/drill-jdbc-all-1.17.0-SNAPSHOT.jar
>  to check if it is assignable to java.sql.Driver. Reason: 
> java.lang.NoClassDefFoundError: oadd/io/protostuff/Schema
> 2019-11-28 15:22:53,468 [Thread-0] INFO  
> net.sourceforge.squirrel_sql.client.gui.db.DriverInternalFrame  - Failed to 
> load 
> oadd.org.apache.drill.exec.proto.SchemaUserBitShared$ExceptionWrapper$MessageSchema
>  in 
> /Users/vova/work/drill/distribution/target/apache-drill-1.17.0-SNAPSHOT/apache-drill-1.17.0-SNAPSHOT/jars/jdbc-driver/drill-jdbc-all-1.17.0-SNAPSHOT.jar
>  to check if it is assignable to java.sql.Driver. Reason: 
> java.lang.NoClassDefFoundError: oadd/io/protostuff/Schema
> 2019-11-28 15:22:53,468 [Thread-0] INFO  
> net.sourceforge.squirrel_sql.client.gui.db.DriverInternalFrame  - Failed to 
> load 
> oadd.org.apache.drill.exec.proto.SchemaUserProtos$CreatePreparedStatementReq$MessageSchema
>  in 
> /Users/vova/work/drill/distribution/target/apache-drill-1.17.0-SNAPSHOT/apache-drill-1.17.0-SNAPSHOT/jars/jdbc-driver/drill-jdbc-all-1.17.0-SNAPSHOT.jar
>  to check if it is assignable to java.sql.Driver. Reason: 
> java.lang.NoClassDefFoundError: oadd/io/protostuff/Schema
> 2019-11-28 15:22:53,470 [Thread-0] INFO  
> net.sourceforge.squirrel_sql.client.gui.db.DriverInternalFrame  - Failed to 
> load 
> oadd.org.apache.drill.exec.proto.SchemaUserBitShared$MetricValue$MessageSchema
>  in 
> /Users/vova/work/drill/distribution/target/apache-drill-1.17.0-SNAPSHOT/apache-drill-1.17.0-SNAPSHOT/jars/jdbc-driver/drill-jdbc-all-1.17.0-SNAPSHOT.jar
>  to check if it is assignable to java.sql.Driver. Reason: 
> java.lang.NoClassDefFoundError: oadd/io/protostuff/Schema
> 2019-11-28 15:22:53,471 [Thread-0] INFO  
> net.sourceforge.squirrel_sql.client.gui.db.DriverInternalFrame  - Failed to 
> load 
> oadd.org.apache.drill.exec.proto.SchemaGeneralRPCProtos$Ack$MessageSchema in 
> /Users/vova/work/drill/distribution/target/apache-drill-1.17.0-SNAPSHOT/apache-drill-1.17.0-SNAPSHOT/jars/jdbc-driver/drill-jdbc-all-1.17.0-SNAPSHOT.jar
>  to check if it is assignable to java.sql.Driver. Reason: 
> java.lang.NoClassDefFoundError: oadd/io/protostuff/Schema
>

[jira] [Updated] (DRILL-7460) Exclude classes which cannot be loaded from JDBC Driver

2019-12-03 Thread Vova Vysotskyi (Jira) 


 [ 
https://issues.apache.org/jira/browse/DRILL-7460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vova Vysotskyi updated DRILL-7460:
--
Fix Version/s: 1.17.0

> Exclude classes which cannot be loaded from JDBC Driver 
> 
>
> Key: DRILL-7460
> URL: https://issues.apache.org/jira/browse/DRILL-7460
> Project: Apache Drill
>  Issue Type: Bug
>Affects Versions: 1.16.0
>Reporter: Vova Vysotskyi
>Assignee: Vova Vysotskyi
>Priority: Minor
> Fix For: 1.17.0
>
>
> When using drill-jdbc-driver with SQuirreLSQL, it logs the following errors:
> {noformat}
> 2019-11-28 15:22:53,406 [Thread-0] DEBUG 
> net.sourceforge.squirrel_sql.client.gui.db.DriverInternalFrame  - 
> getFileFromUrl: 
> fileUri.getPath()=/Users/vova/work/drill/distribution/target/apache-drill-1.17.0-SNAPSHOT/apache-drill-1.17.0-SNAPSHOT/jars/jdbc-driver/drill-jdbc-all-1.17.0-SNAPSHOT.jar
> 2019-11-28 15:22:53,431 [Thread-0] INFO  
> net.sourceforge.squirrel_sql.client.gui.db.DriverInternalFrame  - Failed to 
> load oadd.org.apache.drill.common.util.RepeatTestRule in 
> /Users/vova/work/drill/distribution/target/apache-drill-1.17.0-SNAPSHOT/apache-drill-1.17.0-SNAPSHOT/jars/jdbc-driver/drill-jdbc-all-1.17.0-SNAPSHOT.jar
>  to check if it is assignable to java.sql.Driver. Reason: 
> java.lang.NoClassDefFoundError: org/junit/rules/TestRule
> 2019-11-28 15:22:53,432 [Thread-0] INFO  
> net.sourceforge.squirrel_sql.client.gui.db.DriverInternalFrame  - Failed to 
> load oadd.org.apache.drill.common.util.RepeatTestRule$RepeatStatement in 
> /Users/vova/work/drill/distribution/target/apache-drill-1.17.0-SNAPSHOT/apache-drill-1.17.0-SNAPSHOT/jars/jdbc-driver/drill-jdbc-all-1.17.0-SNAPSHOT.jar
>  to check if it is assignable to java.sql.Driver. Reason: 
> java.lang.NoClassDefFoundError: org/junit/runners/model/Statement
> 2019-11-28 15:22:53,458 [Thread-0] INFO  
> net.sourceforge.squirrel_sql.client.gui.db.DriverInternalFrame  - Failed to 
> load 
> oadd.org.apache.drill.exec.proto.SchemaGeneralRPCProtos$CompleteRpcMessage$BuilderSchema
>  in 
> /Users/vova/work/drill/distribution/target/apache-drill-1.17.0-SNAPSHOT/apache-drill-1.17.0-SNAPSHOT/jars/jdbc-driver/drill-jdbc-all-1.17.0-SNAPSHOT.jar
>  to check if it is assignable to java.sql.Driver. Reason: 
> java.lang.NoClassDefFoundError: oadd/io/protostuff/Schema
> 2019-11-28 15:22:53,458 [Thread-0] INFO  
> net.sourceforge.squirrel_sql.client.gui.db.DriverInternalFrame  - Failed to 
> load 
> oadd.org.apache.drill.exec.proto.SchemaBitControl$BitControlHandshake$BuilderSchema
>  in 
> /Users/vova/work/drill/distribution/target/apache-drill-1.17.0-SNAPSHOT/apache-drill-1.17.0-SNAPSHOT/jars/jdbc-driver/drill-jdbc-all-1.17.0-SNAPSHOT.jar
>  to check if it is assignable to java.sql.Driver. Reason: 
> java.lang.NoClassDefFoundError: oadd/io/protostuff/Schema
> 2019-11-28 15:22:53,468 [Thread-0] INFO  
> net.sourceforge.squirrel_sql.client.gui.db.DriverInternalFrame  - Failed to 
> load 
> oadd.org.apache.drill.exec.proto.SchemaUserBitShared$ExceptionWrapper$MessageSchema
>  in 
> /Users/vova/work/drill/distribution/target/apache-drill-1.17.0-SNAPSHOT/apache-drill-1.17.0-SNAPSHOT/jars/jdbc-driver/drill-jdbc-all-1.17.0-SNAPSHOT.jar
>  to check if it is assignable to java.sql.Driver. Reason: 
> java.lang.NoClassDefFoundError: oadd/io/protostuff/Schema
> 2019-11-28 15:22:53,468 [Thread-0] INFO  
> net.sourceforge.squirrel_sql.client.gui.db.DriverInternalFrame  - Failed to 
> load 
> oadd.org.apache.drill.exec.proto.SchemaUserProtos$CreatePreparedStatementReq$MessageSchema
>  in 
> /Users/vova/work/drill/distribution/target/apache-drill-1.17.0-SNAPSHOT/apache-drill-1.17.0-SNAPSHOT/jars/jdbc-driver/drill-jdbc-all-1.17.0-SNAPSHOT.jar
>  to check if it is assignable to java.sql.Driver. Reason: 
> java.lang.NoClassDefFoundError: oadd/io/protostuff/Schema
> 2019-11-28 15:22:53,470 [Thread-0] INFO  
> net.sourceforge.squirrel_sql.client.gui.db.DriverInternalFrame  - Failed to 
> load 
> oadd.org.apache.drill.exec.proto.SchemaUserBitShared$MetricValue$MessageSchema
>  in 
> /Users/vova/work/drill/distribution/target/apache-drill-1.17.0-SNAPSHOT/apache-drill-1.17.0-SNAPSHOT/jars/jdbc-driver/drill-jdbc-all-1.17.0-SNAPSHOT.jar
>  to check if it is assignable to java.sql.Driver. Reason: 
> java.lang.NoClassDefFoundError: oadd/io/protostuff/Schema
> 2019-11-28 15:22:53,471 [Thread-0] INFO  
> net.sourceforge.squirrel_sql.client.gui.db.DriverInternalFrame  - Failed to 
> load 
> oadd.org.apache.drill.exec.proto.SchemaGeneralRPCProtos$Ack$MessageSchema in 
> /Users/vova/work/drill/distribution/target/apache-drill-1.17.0-SNAPSHOT/apache-drill-1.17.0-SNAPSHOT/jars/jdbc-driver/drill-jdbc-all-1.17.0-SNAPSHOT.jar
>  to check if it is assignable to java.sql.Driver. Reason: 
> java.lang.NoClassDefFoundError:

[jira] [Commented] (DRILL-7359) Add support for DICT type in RowSet Framework

2019-12-03 Thread ASF GitHub Bot (Jira) 


[ 
https://issues.apache.org/jira/browse/DRILL-7359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986824#comment-16986824
 ] 

ASF GitHub Bot commented on DRILL-7359:
---

KazydubB commented on pull request #1870: DRILL-7359: Add support for DICT type 
in RowSet Framework
URL: https://github.com/apache/drill/pull/1870#discussion_r353132642
 
 

 ##
 File path: 
exec/vector/src/main/java/org/apache/drill/exec/vector/accessor/reader/UnionReaderImpl.java
 ##
 @@ -268,4 +274,51 @@ public String getAsString() {
 }
 return requireReader(type).getAsString();
   }
+
+  private UnionReaderImpl getNullReader() {
+AbstractObjectReader[] nullVariants = new 
AbstractObjectReader[variants.length];
+for (int i = 0; i < variants.length; i++) {
+  nullVariants[i] = variants[i].createNullReader();
+}
+return new NullUnionReader(schema(), unionAccessor, nullVariants);
+  }
+
+  private static class NullUnionReader extends UnionReaderImpl {
+
+private NullUnionReader(ColumnMetadata metadata, VectorAccessor va, 
AbstractObjectReader[] variants) {
 
 Review comment:
   Regarding points of your suggestion:
   - `DictReaderImpl` has `NullStateReaders.REQUIRED_STATE_READER` 
`NullStateReader`, i.e. it can't be `NULL` (like `MapReader`).
   - `DICT` can't have `NULL` key as it is `REQUIRED`.
   - If the `DICT` has a key, but the value is `NULL`, then `isNull()` for the 
value returns `true` - yes.
   - in the (tricky) case of a missing key from the `DICT`, the latter approach 
was taken - there is no differentiation between missing key and `NULL` value. 
(The new `Null*Reader`s are introduced to be used in the case of a missing key 
- their `isNull()` always returns `true`; note, that in case when a key is 
present and the value is `NULL`, current (non-null) readers' `isNull()` will 
return `true` for that particular position, as corresponding value reader keeps 
track of nullability). The former approach was discarded to ensure uniform 
behavior without a chance of getting `NPE` or making `null`-checks.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Add support for DICT type in RowSet Framework
> -
>
> Key: DRILL-7359
> URL: https://issues.apache.org/jira/browse/DRILL-7359
> Project: Apache Drill
>  Issue Type: New Feature
>Reporter: Bohdan Kazydub
>Assignee: Bohdan Kazydub
>Priority: Major
> Fix For: 1.17.0
>
>
> Add support for new DICT data type (see DRILL-7096) in RowSet Framework



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (DRILL-7393) Revisit Drill tests to ensure that patching is executed before any test run

2019-12-03 Thread ASF GitHub Bot (Jira) 


[ 
https://issues.apache.org/jira/browse/DRILL-7393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986789#comment-16986789
 ] 

ASF GitHub Bot commented on DRILL-7393:
---

vvysotskyi commented on issue #1910: DRILL-7393: Revisit Drill tests to ensure 
that patching is executed b…
URL: https://github.com/apache/drill/pull/1910#issuecomment-56801
 
 
   @paul-rogers, all these approaches require a large amount of time. What do 
you think about @arina-ielchiieva's suggestion from this comment 
https://github.com/apache/drill/pull/1910#issuecomment-560383657 to unblock the 
release?
   
   Discussing the approach on how to fix the general problem and implementing 
it may be done after the release.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Revisit Drill tests to ensure that patching is executed before any test run
> ---
>
> Key: DRILL-7393
> URL: https://issues.apache.org/jira/browse/DRILL-7393
> Project: Apache Drill
>  Issue Type: Task
>Affects Versions: 1.16.0, 1.17.0
>Reporter: Arina Ielchiieva
>Assignee: Anton Gozhiy
>Priority: Major
>  Labels: ready-to-commit
>
> Apache Drill patches some Protobuf and Guava classes (see GuavaPatcher, 
> ProtobufPatcher), patching should be done before classes to be patched are 
> loaded. That's why this operation is executed in static block in Drillbit 
> class.
> Some tests in java-exec module use Drillbit class, some extend DrillTest 
> class, both of them patch Guava. But there are some tests that do not call 
> patcher but load classes to be patched. For example, 
> {{org.apache.drill.exec.sql.TestSqlBracketlessSyntax}} loads Guava 
> Preconditions class. If such tests run before tests that require patching, 
> tests run will fail since patching won't be successful. Patchers code does 
> not fail application if patching was not complete, just logs warning 
> ({{logger.warn("Unable to patch Guava classes.", e);}}), so sometimes it hard 
> to identify unit tests failure root cause.
> We need to revisit all Drill tests to ensure that all of them extend common 
> test base class which patchers Protobuf and Guava classes in static block. 
> Also refactor Patcher classes to have assert to fail if patching fails during 
> unit testing if there are any problems.
> After all tests are revised, we can remove {{metastore-test}} execution from 
> main.xml in {{maven-surefire-plugin}} which was added to ensure that all 
> Metastore tests run in a separate JVM where patching is done in first place 
> since Iceberg Metastore heavily depends on patched Guava Preconditions class.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)