[jira] [Commented] (DRILL-6588) System table columns incorrectly marked as non-nullable
[
https://issues.apache.org/jira/browse/DRILL-6588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16544284#comment-16544284
]
ASF GitHub Bot commented on DRILL-6588:
---
sohami closed pull request #1371: DRILL-6588: Make Sys tables of nullable
datatypes
URL: https://github.com/apache/drill/pull/1371
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git
a/exec/java-exec/src/main/java/org/apache/drill/exec/store/RecordDataType.java
b/exec/java-exec/src/main/java/org/apache/drill/exec/store/RecordDataType.java
index 017e257f20e..1f5b93838f3 100644
---
a/exec/java-exec/src/main/java/org/apache/drill/exec/store/RecordDataType.java
+++
b/exec/java-exec/src/main/java/org/apache/drill/exec/store/RecordDataType.java
@@ -17,22 +17,23 @@
*/
package org.apache.drill.exec.store;
-import com.google.common.collect.Lists;
+import java.util.AbstractMap.SimpleImmutableEntry;
+import java.util.ArrayList;
+import java.util.List;
+
import org.apache.calcite.rel.type.RelDataType;
import org.apache.calcite.rel.type.RelDataTypeFactory;
import org.apache.calcite.sql.type.SqlTypeName;
-import java.util.List;
-
/**
* RecordDataType defines names and data types of columns in a static drill
table.
*/
public abstract class RecordDataType {
/**
- * @return the {@link org.apache.calcite.sql.type.SqlTypeName} of columns in
the table
+ * @return the {@link org.apache.calcite.sql.type.SqlTypeName} of columns in
the table as a pair with its nullability
*/
- public abstract List getFieldSqlTypeNames();
+ public abstract List>
getFieldSqlTypeNames();
/**
* @return the column names in the table
@@ -47,17 +48,21 @@
* @return the constructed type
*/
public final RelDataType getRowType(RelDataTypeFactory factory) {
-final List types = getFieldSqlTypeNames();
+final List> types =
getFieldSqlTypeNames();
final List names = getFieldNames();
-final List fields = Lists.newArrayList();
-for (final SqlTypeName typeName : types) {
+final List fields = new ArrayList<>();
+for (SimpleImmutableEntry sqlTypePair : types) {
+ final SqlTypeName typeName = sqlTypePair.getKey();
+ final RelDataType tempDataType;
switch (typeName) {
case VARCHAR:
- fields.add(factory.createSqlType(typeName, Integer.MAX_VALUE));
+ tempDataType = factory.createSqlType(typeName, Integer.MAX_VALUE);
break;
default:
- fields.add(factory.createSqlType(typeName));
+ tempDataType = factory.createSqlType(typeName);
}
+ //Add [Non]Nullable RelDataType
+ fields.add(factory.createTypeWithNullability(tempDataType,
sqlTypePair.getValue()));
}
return factory.createStructType(fields, names);
}
diff --git
a/exec/java-exec/src/main/java/org/apache/drill/exec/store/pojo/NonNullable.java
b/exec/java-exec/src/main/java/org/apache/drill/exec/store/pojo/NonNullable.java
new file mode 100644
index 000..18fca7cf83e
--- /dev/null
+++
b/exec/java-exec/src/main/java/org/apache/drill/exec/store/pojo/NonNullable.java
@@ -0,0 +1,32 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.drill.exec.store.pojo;
+
+import java.lang.annotation.ElementType;
+import java.lang.annotation.Retention;
+import java.lang.annotation.RetentionPolicy;
+import java.lang.annotation.Target;
+
+/**
+ * Indicates NonNullable nature
+ */
+@Retention(RetentionPolicy.RUNTIME)
+@Target(ElementType.FIELD)
+public @interface NonNullable {
+ boolean value() default true;
+}
diff --git
a/exec/java-exec/src/main/java/org/apache/drill/exec/store/pojo/PojoDataType.java
b/exec/java-exec/src/main/java/org/apache/drill/exec/store/pojo/PojoDataType.java
index 7d8a8e49202..e91287925b1 100644
---
a/exec/java-exec/src/main/java/org/apache/drill/exec/store/pojo/PojoDataType.java
+++
b/exec/java-exec/src/main/java/org/apache/drill/exec/store/pojo/PojoDataType.java
[jira] [Commented] (DRILL-6588) System table columns incorrectly marked as non-nullable
[
https://issues.apache.org/jira/browse/DRILL-6588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16543665#comment-16543665
]
ASF GitHub Bot commented on DRILL-6588:
---
arina-ielchiieva commented on a change in pull request #1371: DRILL-6588: Make
Sys tables of nullable datatypes
URL: https://github.com/apache/drill/pull/1371#discussion_r202464481
##
File path:
exec/java-exec/src/test/java/org/apache/drill/exec/store/sys/TestSystemTable.java
##
@@ -90,4 +92,11 @@ public void testProfilesLimitPushDown() throws Exception {
String numFilesPattern = "maxRecordsToRead=10";
testPlanMatchingPatterns(query, new String[] {numFilesPattern}, new
String[] {});
}
+
+ @Test
+ public void testColumnNullability() throws Exception {
+String query = " select distinct is_nullable, count(*) from
INFORMATION_SCHEMA.`COLUMNS` where table_schema = 'sys' group by is_nullable";
Review comment:
`" select` -> please remove space
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> System table columns incorrectly marked as non-nullable
>
>
> Key: DRILL-6588
> URL: https://issues.apache.org/jira/browse/DRILL-6588
> Project: Apache Drill
> Issue Type: Bug
> Components: Metadata
>Affects Versions: 1.13.0
>Reporter: Aman Sinha
>Assignee: Kunal Khatua
>Priority: Major
> Fix For: 1.14.0
>
>
> System table columns can contain null values but they are incorrectly marked
> as non-nullable as shown in example table below:
> {noformat}
> 0: jdbc:drill:drillbit=10.10.10.191> describe sys.boot;
> +---++--+
> | COLUMN_NAME | DATA_TYPE | IS_NULLABLE |
> +---++--+
> | name | CHARACTER VARYING | NO |
> | kind | CHARACTER VARYING | NO |
> | accessibleScopes | CHARACTER VARYING | NO |
> | optionScope | CHARACTER VARYING | NO |
> | status | CHARACTER VARYING | NO |
> | num_val | BIGINT | NO |
> | string_val | CHARACTER VARYING | NO |
> | bool_val | BOOLEAN | NO |
> | float_val | DOUBLE | NO |
> +---++--+{noformat}
>
> Note that several columns are nulls:
> {noformat}
> +---+--+--+-++-++--+---+
> | name | kind |
> accessibleScopes | optionScope | status | num_val | string_val | bool_val |
> float_val |
> +---+--+--+-++-++--+---+
> drill.exec.options.exec.udf.enable_dynamic_support | BOOLEAN | BOOT | BOOT |
> BOOT | null | null | true | null |{noformat}
>
> Because of the not-null metadata, the predicates on these tables such as
> `WHERE IS NULL` evaluate to FALSE which is incorrect.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6588) System table columns incorrectly marked as non-nullable
[
https://issues.apache.org/jira/browse/DRILL-6588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16543667#comment-16543667
]
ASF GitHub Bot commented on DRILL-6588:
---
arina-ielchiieva commented on issue #1371: DRILL-6588: Make Sys tables of
nullable datatypes
URL: https://github.com/apache/drill/pull/1371#issuecomment-404945249
Once minor comment to remove space, also please squash the commits. +1, LGTM.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> System table columns incorrectly marked as non-nullable
>
>
> Key: DRILL-6588
> URL: https://issues.apache.org/jira/browse/DRILL-6588
> Project: Apache Drill
> Issue Type: Bug
> Components: Metadata
>Affects Versions: 1.13.0
>Reporter: Aman Sinha
>Assignee: Kunal Khatua
>Priority: Major
> Fix For: 1.14.0
>
>
> System table columns can contain null values but they are incorrectly marked
> as non-nullable as shown in example table below:
> {noformat}
> 0: jdbc:drill:drillbit=10.10.10.191> describe sys.boot;
> +---++--+
> | COLUMN_NAME | DATA_TYPE | IS_NULLABLE |
> +---++--+
> | name | CHARACTER VARYING | NO |
> | kind | CHARACTER VARYING | NO |
> | accessibleScopes | CHARACTER VARYING | NO |
> | optionScope | CHARACTER VARYING | NO |
> | status | CHARACTER VARYING | NO |
> | num_val | BIGINT | NO |
> | string_val | CHARACTER VARYING | NO |
> | bool_val | BOOLEAN | NO |
> | float_val | DOUBLE | NO |
> +---++--+{noformat}
>
> Note that several columns are nulls:
> {noformat}
> +---+--+--+-++-++--+---+
> | name | kind |
> accessibleScopes | optionScope | status | num_val | string_val | bool_val |
> float_val |
> +---+--+--+-++-++--+---+
> drill.exec.options.exec.udf.enable_dynamic_support | BOOLEAN | BOOT | BOOT |
> BOOT | null | null | true | null |{noformat}
>
> Because of the not-null metadata, the predicates on these tables such as
> `WHERE IS NULL` evaluate to FALSE which is incorrect.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6588) System table columns incorrectly marked as non-nullable
[
https://issues.apache.org/jira/browse/DRILL-6588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16543542#comment-16543542
]
ASF GitHub Bot commented on DRILL-6588:
---
kkhatua commented on issue #1371: DRILL-6588: Make Sys tables of nullable
datatypes
URL: https://github.com/apache/drill/pull/1371#issuecomment-404909514
Added a unit test that groups and counts the number of IS_NULLABLE values.
Originally, the query would have returned only 1 group (`IS_NULLABLE = false`).
After the patch, there are 2 records for both boolean values.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> System table columns incorrectly marked as non-nullable
>
>
> Key: DRILL-6588
> URL: https://issues.apache.org/jira/browse/DRILL-6588
> Project: Apache Drill
> Issue Type: Bug
> Components: Metadata
>Affects Versions: 1.13.0
>Reporter: Aman Sinha
>Assignee: Kunal Khatua
>Priority: Major
> Fix For: 1.14.0
>
>
> System table columns can contain null values but they are incorrectly marked
> as non-nullable as shown in example table below:
> {noformat}
> 0: jdbc:drill:drillbit=10.10.10.191> describe sys.boot;
> +---++--+
> | COLUMN_NAME | DATA_TYPE | IS_NULLABLE |
> +---++--+
> | name | CHARACTER VARYING | NO |
> | kind | CHARACTER VARYING | NO |
> | accessibleScopes | CHARACTER VARYING | NO |
> | optionScope | CHARACTER VARYING | NO |
> | status | CHARACTER VARYING | NO |
> | num_val | BIGINT | NO |
> | string_val | CHARACTER VARYING | NO |
> | bool_val | BOOLEAN | NO |
> | float_val | DOUBLE | NO |
> +---++--+{noformat}
>
> Note that several columns are nulls:
> {noformat}
> +---+--+--+-++-++--+---+
> | name | kind |
> accessibleScopes | optionScope | status | num_val | string_val | bool_val |
> float_val |
> +---+--+--+-++-++--+---+
> drill.exec.options.exec.udf.enable_dynamic_support | BOOLEAN | BOOT | BOOT |
> BOOT | null | null | true | null |{noformat}
>
> Because of the not-null metadata, the predicates on these tables such as
> `WHERE IS NULL` evaluate to FALSE which is incorrect.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6588) System table columns incorrectly marked as non-nullable
[
https://issues.apache.org/jira/browse/DRILL-6588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16542862#comment-16542862
]
ASF GitHub Bot commented on DRILL-6588:
---
arina-ielchiieva edited a comment on issue #1371: DRILL-6588: Make Sys tables
of nullable datatypes
URL: https://github.com/apache/drill/pull/1371#issuecomment-404765907
@kkhatua could you please add unit tests, similar to the queries you showed
in description.
You can add with `@Unlikely` but still it's better to have them.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> System table columns incorrectly marked as non-nullable
>
>
> Key: DRILL-6588
> URL: https://issues.apache.org/jira/browse/DRILL-6588
> Project: Apache Drill
> Issue Type: Bug
> Components: Metadata
>Affects Versions: 1.13.0
>Reporter: Aman Sinha
>Assignee: Kunal Khatua
>Priority: Major
> Fix For: 1.14.0
>
>
> System table columns can contain null values but they are incorrectly marked
> as non-nullable as shown in example table below:
> {noformat}
> 0: jdbc:drill:drillbit=10.10.10.191> describe sys.boot;
> +---++--+
> | COLUMN_NAME | DATA_TYPE | IS_NULLABLE |
> +---++--+
> | name | CHARACTER VARYING | NO |
> | kind | CHARACTER VARYING | NO |
> | accessibleScopes | CHARACTER VARYING | NO |
> | optionScope | CHARACTER VARYING | NO |
> | status | CHARACTER VARYING | NO |
> | num_val | BIGINT | NO |
> | string_val | CHARACTER VARYING | NO |
> | bool_val | BOOLEAN | NO |
> | float_val | DOUBLE | NO |
> +---++--+{noformat}
>
> Note that several columns are nulls:
> {noformat}
> +---+--+--+-++-++--+---+
> | name | kind |
> accessibleScopes | optionScope | status | num_val | string_val | bool_val |
> float_val |
> +---+--+--+-++-++--+---+
> drill.exec.options.exec.udf.enable_dynamic_support | BOOLEAN | BOOT | BOOT |
> BOOT | null | null | true | null |{noformat}
>
> Because of the not-null metadata, the predicates on these tables such as
> `WHERE IS NULL` evaluate to FALSE which is incorrect.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6588) System table columns incorrectly marked as non-nullable
[
https://issues.apache.org/jira/browse/DRILL-6588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16542713#comment-16542713
]
ASF GitHub Bot commented on DRILL-6588:
---
arina-ielchiieva commented on issue #1371: DRILL-6588: Make Sys tables of
nullable datatypes
URL: https://github.com/apache/drill/pull/1371#issuecomment-404765907
@kkhatua could you please add unit tests, similar to the queries you showed
in description.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> System table columns incorrectly marked as non-nullable
>
>
> Key: DRILL-6588
> URL: https://issues.apache.org/jira/browse/DRILL-6588
> Project: Apache Drill
> Issue Type: Bug
> Components: Metadata
>Affects Versions: 1.13.0
>Reporter: Aman Sinha
>Assignee: Kunal Khatua
>Priority: Major
> Fix For: 1.14.0
>
>
> System table columns can contain null values but they are incorrectly marked
> as non-nullable as shown in example table below:
> {noformat}
> 0: jdbc:drill:drillbit=10.10.10.191> describe sys.boot;
> +---++--+
> | COLUMN_NAME | DATA_TYPE | IS_NULLABLE |
> +---++--+
> | name | CHARACTER VARYING | NO |
> | kind | CHARACTER VARYING | NO |
> | accessibleScopes | CHARACTER VARYING | NO |
> | optionScope | CHARACTER VARYING | NO |
> | status | CHARACTER VARYING | NO |
> | num_val | BIGINT | NO |
> | string_val | CHARACTER VARYING | NO |
> | bool_val | BOOLEAN | NO |
> | float_val | DOUBLE | NO |
> +---++--+{noformat}
>
> Note that several columns are nulls:
> {noformat}
> +---+--+--+-++-++--+---+
> | name | kind |
> accessibleScopes | optionScope | status | num_val | string_val | bool_val |
> float_val |
> +---+--+--+-++-++--+---+
> drill.exec.options.exec.udf.enable_dynamic_support | BOOLEAN | BOOT | BOOT |
> BOOT | null | null | true | null |{noformat}
>
> Because of the not-null metadata, the predicates on these tables such as
> `WHERE IS NULL` evaluate to FALSE which is incorrect.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6588) System table columns incorrectly marked as non-nullable
[
https://issues.apache.org/jira/browse/DRILL-6588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16542166#comment-16542166
]
ASF GitHub Bot commented on DRILL-6588:
---
kkhatua commented on issue #1371: DRILL-6588: Make Sys tables of nullable
datatypes
URL: https://github.com/apache/drill/pull/1371#issuecomment-404638107
@arina-ielchiieva / @paul-rogers Done the final changes.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> System table columns incorrectly marked as non-nullable
>
>
> Key: DRILL-6588
> URL: https://issues.apache.org/jira/browse/DRILL-6588
> Project: Apache Drill
> Issue Type: Bug
> Components: Metadata
>Affects Versions: 1.13.0
>Reporter: Aman Sinha
>Assignee: Kunal Khatua
>Priority: Major
> Fix For: 1.14.0
>
>
> System table columns can contain null values but they are incorrectly marked
> as non-nullable as shown in example table below:
> {noformat}
> 0: jdbc:drill:drillbit=10.10.10.191> describe sys.boot;
> +---++--+
> | COLUMN_NAME | DATA_TYPE | IS_NULLABLE |
> +---++--+
> | name | CHARACTER VARYING | NO |
> | kind | CHARACTER VARYING | NO |
> | accessibleScopes | CHARACTER VARYING | NO |
> | optionScope | CHARACTER VARYING | NO |
> | status | CHARACTER VARYING | NO |
> | num_val | BIGINT | NO |
> | string_val | CHARACTER VARYING | NO |
> | bool_val | BOOLEAN | NO |
> | float_val | DOUBLE | NO |
> +---++--+{noformat}
>
> Note that several columns are nulls:
> {noformat}
> +---+--+--+-++-++--+---+
> | name | kind |
> accessibleScopes | optionScope | status | num_val | string_val | bool_val |
> float_val |
> +---+--+--+-++-++--+---+
> drill.exec.options.exec.udf.enable_dynamic_support | BOOLEAN | BOOT | BOOT |
> BOOT | null | null | true | null |{noformat}
>
> Because of the not-null metadata, the predicates on these tables such as
> `WHERE IS NULL` evaluate to FALSE which is incorrect.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6588) System table columns incorrectly marked as non-nullable
[
https://issues.apache.org/jira/browse/DRILL-6588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16541309#comment-16541309
]
ASF GitHub Bot commented on DRILL-6588:
---
arina-ielchiieva commented on a change in pull request #1371: DRILL-6588: Make
Sys tables of nullable datatypes
URL: https://github.com/apache/drill/pull/1371#discussion_r201948032
##
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/store/RecordDataType.java
##
@@ -47,17 +48,22 @@
* @return the constructed type
*/
public final RelDataType getRowType(RelDataTypeFactory factory) {
-final List types = getFieldSqlTypeNames();
+final List> types =
getFieldSqlTypeNames();
Review comment:
Ok, don't have strong opinion about this right now, so you may leave as is.
But I would prefer Java native implementation in this case:
```
AbstractMap.SimpleImmutableEntry entry
= new AbstractMap.SimpleImmutableEntry<>(1, "one");
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> System table columns incorrectly marked as non-nullable
>
>
> Key: DRILL-6588
> URL: https://issues.apache.org/jira/browse/DRILL-6588
> Project: Apache Drill
> Issue Type: Bug
> Components: Metadata
>Affects Versions: 1.13.0
>Reporter: Aman Sinha
>Assignee: Kunal Khatua
>Priority: Major
> Fix For: 1.14.0
>
>
> System table columns can contain null values but they are incorrectly marked
> as non-nullable as shown in example table below:
> {noformat}
> 0: jdbc:drill:drillbit=10.10.10.191> describe sys.boot;
> +---++--+
> | COLUMN_NAME | DATA_TYPE | IS_NULLABLE |
> +---++--+
> | name | CHARACTER VARYING | NO |
> | kind | CHARACTER VARYING | NO |
> | accessibleScopes | CHARACTER VARYING | NO |
> | optionScope | CHARACTER VARYING | NO |
> | status | CHARACTER VARYING | NO |
> | num_val | BIGINT | NO |
> | string_val | CHARACTER VARYING | NO |
> | bool_val | BOOLEAN | NO |
> | float_val | DOUBLE | NO |
> +---++--+{noformat}
>
> Note that several columns are nulls:
> {noformat}
> +---+--+--+-++-++--+---+
> | name | kind |
> accessibleScopes | optionScope | status | num_val | string_val | bool_val |
> float_val |
> +---+--+--+-++-++--+---+
> drill.exec.options.exec.udf.enable_dynamic_support | BOOLEAN | BOOT | BOOT |
> BOOT | null | null | true | null |{noformat}
>
> Because of the not-null metadata, the predicates on these tables such as
> `WHERE IS NULL` evaluate to FALSE which is incorrect.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6588) System table columns incorrectly marked as non-nullable
[
https://issues.apache.org/jira/browse/DRILL-6588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16541310#comment-16541310
]
ASF GitHub Bot commented on DRILL-6588:
---
arina-ielchiieva commented on a change in pull request #1371: DRILL-6588: Make
Sys tables of nullable datatypes
URL: https://github.com/apache/drill/pull/1371#discussion_r201946028
##
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/store/pojo/PojoDataType.java
##
@@ -34,7 +37,7 @@
public class PojoDataType extends RecordDataType {
// private static final org.slf4j.Logger logger =
org.slf4j.LoggerFactory.getLogger(PojoDataType.class);
- private final List types = Lists.newArrayList();
+ private final List> types =
Lists.newArrayList();
Review comment:
Please don't use guava, use `new ArrayList<>()`
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> System table columns incorrectly marked as non-nullable
>
>
> Key: DRILL-6588
> URL: https://issues.apache.org/jira/browse/DRILL-6588
> Project: Apache Drill
> Issue Type: Bug
> Components: Metadata
>Affects Versions: 1.13.0
>Reporter: Aman Sinha
>Assignee: Kunal Khatua
>Priority: Major
> Fix For: 1.14.0
>
>
> System table columns can contain null values but they are incorrectly marked
> as non-nullable as shown in example table below:
> {noformat}
> 0: jdbc:drill:drillbit=10.10.10.191> describe sys.boot;
> +---++--+
> | COLUMN_NAME | DATA_TYPE | IS_NULLABLE |
> +---++--+
> | name | CHARACTER VARYING | NO |
> | kind | CHARACTER VARYING | NO |
> | accessibleScopes | CHARACTER VARYING | NO |
> | optionScope | CHARACTER VARYING | NO |
> | status | CHARACTER VARYING | NO |
> | num_val | BIGINT | NO |
> | string_val | CHARACTER VARYING | NO |
> | bool_val | BOOLEAN | NO |
> | float_val | DOUBLE | NO |
> +---++--+{noformat}
>
> Note that several columns are nulls:
> {noformat}
> +---+--+--+-++-++--+---+
> | name | kind |
> accessibleScopes | optionScope | status | num_val | string_val | bool_val |
> float_val |
> +---+--+--+-++-++--+---+
> drill.exec.options.exec.udf.enable_dynamic_support | BOOLEAN | BOOT | BOOT |
> BOOT | null | null | true | null |{noformat}
>
> Because of the not-null metadata, the predicates on these tables such as
> `WHERE IS NULL` evaluate to FALSE which is incorrect.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6588) System table columns incorrectly marked as non-nullable
[
https://issues.apache.org/jira/browse/DRILL-6588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16541311#comment-16541311
]
ASF GitHub Bot commented on DRILL-6588:
---
arina-ielchiieva commented on a change in pull request #1371: DRILL-6588: Make
Sys tables of nullable datatypes
URL: https://github.com/apache/drill/pull/1371#discussion_r201946174
##
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/BitToUserConnectionIterator.java
##
@@ -99,14 +102,18 @@ public void remove() {
public static class ConnectionInfo {
public String user;
+@Nonnull
Review comment:
+1 for @NotNullable
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> System table columns incorrectly marked as non-nullable
>
>
> Key: DRILL-6588
> URL: https://issues.apache.org/jira/browse/DRILL-6588
> Project: Apache Drill
> Issue Type: Bug
> Components: Metadata
>Affects Versions: 1.13.0
>Reporter: Aman Sinha
>Assignee: Kunal Khatua
>Priority: Major
> Fix For: 1.14.0
>
>
> System table columns can contain null values but they are incorrectly marked
> as non-nullable as shown in example table below:
> {noformat}
> 0: jdbc:drill:drillbit=10.10.10.191> describe sys.boot;
> +---++--+
> | COLUMN_NAME | DATA_TYPE | IS_NULLABLE |
> +---++--+
> | name | CHARACTER VARYING | NO |
> | kind | CHARACTER VARYING | NO |
> | accessibleScopes | CHARACTER VARYING | NO |
> | optionScope | CHARACTER VARYING | NO |
> | status | CHARACTER VARYING | NO |
> | num_val | BIGINT | NO |
> | string_val | CHARACTER VARYING | NO |
> | bool_val | BOOLEAN | NO |
> | float_val | DOUBLE | NO |
> +---++--+{noformat}
>
> Note that several columns are nulls:
> {noformat}
> +---+--+--+-++-++--+---+
> | name | kind |
> accessibleScopes | optionScope | status | num_val | string_val | bool_val |
> float_val |
> +---+--+--+-++-++--+---+
> drill.exec.options.exec.udf.enable_dynamic_support | BOOLEAN | BOOT | BOOT |
> BOOT | null | null | true | null |{noformat}
>
> Because of the not-null metadata, the predicates on these tables such as
> `WHERE IS NULL` evaluate to FALSE which is incorrect.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6588) System table columns incorrectly marked as non-nullable
[
https://issues.apache.org/jira/browse/DRILL-6588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16541308#comment-16541308
]
ASF GitHub Bot commented on DRILL-6588:
---
arina-ielchiieva commented on a change in pull request #1371: DRILL-6588: Make
Sys tables of nullable datatypes
URL: https://github.com/apache/drill/pull/1371#discussion_r201945843
##
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/store/pojo/PojoDataType.java
##
@@ -75,13 +81,12 @@ public PojoDataType(Class pojoClass) {
}
@Override
- public List getFieldSqlTypeNames() {
+ public List> getFieldSqlTypeNames() {
Review comment:
`SqlTypeName,Boolean` -> space
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> System table columns incorrectly marked as non-nullable
>
>
> Key: DRILL-6588
> URL: https://issues.apache.org/jira/browse/DRILL-6588
> Project: Apache Drill
> Issue Type: Bug
> Components: Metadata
>Affects Versions: 1.13.0
>Reporter: Aman Sinha
>Assignee: Kunal Khatua
>Priority: Major
> Fix For: 1.14.0
>
>
> System table columns can contain null values but they are incorrectly marked
> as non-nullable as shown in example table below:
> {noformat}
> 0: jdbc:drill:drillbit=10.10.10.191> describe sys.boot;
> +---++--+
> | COLUMN_NAME | DATA_TYPE | IS_NULLABLE |
> +---++--+
> | name | CHARACTER VARYING | NO |
> | kind | CHARACTER VARYING | NO |
> | accessibleScopes | CHARACTER VARYING | NO |
> | optionScope | CHARACTER VARYING | NO |
> | status | CHARACTER VARYING | NO |
> | num_val | BIGINT | NO |
> | string_val | CHARACTER VARYING | NO |
> | bool_val | BOOLEAN | NO |
> | float_val | DOUBLE | NO |
> +---++--+{noformat}
>
> Note that several columns are nulls:
> {noformat}
> +---+--+--+-++-++--+---+
> | name | kind |
> accessibleScopes | optionScope | status | num_val | string_val | bool_val |
> float_val |
> +---+--+--+-++-++--+---+
> drill.exec.options.exec.udf.enable_dynamic_support | BOOLEAN | BOOT | BOOT |
> BOOT | null | null | true | null |{noformat}
>
> Because of the not-null metadata, the predicates on these tables such as
> `WHERE IS NULL` evaluate to FALSE which is incorrect.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6588) System table columns incorrectly marked as non-nullable
[
https://issues.apache.org/jira/browse/DRILL-6588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16541144#comment-16541144
]
ASF GitHub Bot commented on DRILL-6588:
---
paul-rogers commented on a change in pull request #1371: DRILL-6588: Make Sys
tables of nullable datatypes
URL: https://github.com/apache/drill/pull/1371#discussion_r201909347
##
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/BitToUserConnectionIterator.java
##
@@ -99,14 +102,18 @@ public void remove() {
public static class ConnectionInfo {
public String user;
+@Nonnull
Review comment:
Sorry, I was misled by the "javax" package and thought this was a Java
thing. Good catch @arina-ielchiieva.
Since it is not, go ahead and define your own. Today we assume all fields
are nullable, and must keep that default. So, since the default is nullable,
you only need the `@NonNullable` annotation.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> System table columns incorrectly marked as non-nullable
>
>
> Key: DRILL-6588
> URL: https://issues.apache.org/jira/browse/DRILL-6588
> Project: Apache Drill
> Issue Type: Bug
> Components: Metadata
>Affects Versions: 1.13.0
>Reporter: Aman Sinha
>Assignee: Kunal Khatua
>Priority: Major
> Fix For: 1.14.0
>
>
> System table columns can contain null values but they are incorrectly marked
> as non-nullable as shown in example table below:
> {noformat}
> 0: jdbc:drill:drillbit=10.10.10.191> describe sys.boot;
> +---++--+
> | COLUMN_NAME | DATA_TYPE | IS_NULLABLE |
> +---++--+
> | name | CHARACTER VARYING | NO |
> | kind | CHARACTER VARYING | NO |
> | accessibleScopes | CHARACTER VARYING | NO |
> | optionScope | CHARACTER VARYING | NO |
> | status | CHARACTER VARYING | NO |
> | num_val | BIGINT | NO |
> | string_val | CHARACTER VARYING | NO |
> | bool_val | BOOLEAN | NO |
> | float_val | DOUBLE | NO |
> +---++--+{noformat}
>
> Note that several columns are nulls:
> {noformat}
> +---+--+--+-++-++--+---+
> | name | kind |
> accessibleScopes | optionScope | status | num_val | string_val | bool_val |
> float_val |
> +---+--+--+-++-++--+---+
> drill.exec.options.exec.udf.enable_dynamic_support | BOOLEAN | BOOT | BOOT |
> BOOT | null | null | true | null |{noformat}
>
> Because of the not-null metadata, the predicates on these tables such as
> `WHERE IS NULL` evaluate to FALSE which is incorrect.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6588) System table columns incorrectly marked as non-nullable
[
https://issues.apache.org/jira/browse/DRILL-6588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16540961#comment-16540961
]
ASF GitHub Bot commented on DRILL-6588:
---
kkhatua commented on a change in pull request #1371: DRILL-6588: Make Sys
tables of nullable datatypes
URL: https://github.com/apache/drill/pull/1371#discussion_r201883774
##
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/store/RecordDataType.java
##
@@ -47,17 +48,22 @@
* @return the constructed type
*/
public final RelDataType getRowType(RelDataTypeFactory factory) {
-final List types = getFieldSqlTypeNames();
+final List> types =
getFieldSqlTypeNames();
Review comment:
For a one-time usage of creating a nullable RelDataType (since Calcite
doesn't provide a convenient way of doing so), I think a holder class is an
overkill.
I have this option from Calcite:
`https://calcite.apache.org/apidocs/org/apache/calcite/rel/type/RelDataTypeFactory.html#createStructType-java.util.List-`
which also makes use of a list of Map.Entry objects.
So, unless I change Calcite, there is no real value in creating a holder
class.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> System table columns incorrectly marked as non-nullable
>
>
> Key: DRILL-6588
> URL: https://issues.apache.org/jira/browse/DRILL-6588
> Project: Apache Drill
> Issue Type: Bug
> Components: Metadata
>Affects Versions: 1.13.0
>Reporter: Aman Sinha
>Assignee: Kunal Khatua
>Priority: Major
> Fix For: 1.14.0
>
>
> System table columns can contain null values but they are incorrectly marked
> as non-nullable as shown in example table below:
> {noformat}
> 0: jdbc:drill:drillbit=10.10.10.191> describe sys.boot;
> +---++--+
> | COLUMN_NAME | DATA_TYPE | IS_NULLABLE |
> +---++--+
> | name | CHARACTER VARYING | NO |
> | kind | CHARACTER VARYING | NO |
> | accessibleScopes | CHARACTER VARYING | NO |
> | optionScope | CHARACTER VARYING | NO |
> | status | CHARACTER VARYING | NO |
> | num_val | BIGINT | NO |
> | string_val | CHARACTER VARYING | NO |
> | bool_val | BOOLEAN | NO |
> | float_val | DOUBLE | NO |
> +---++--+{noformat}
>
> Note that several columns are nulls:
> {noformat}
> +---+--+--+-++-++--+---+
> | name | kind |
> accessibleScopes | optionScope | status | num_val | string_val | bool_val |
> float_val |
> +---+--+--+-++-++--+---+
> drill.exec.options.exec.udf.enable_dynamic_support | BOOLEAN | BOOT | BOOT |
> BOOT | null | null | true | null |{noformat}
>
> Because of the not-null metadata, the predicates on these tables such as
> `WHERE IS NULL` evaluate to FALSE which is incorrect.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6588) System table columns incorrectly marked as non-nullable
[
https://issues.apache.org/jira/browse/DRILL-6588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16540957#comment-16540957
]
ASF GitHub Bot commented on DRILL-6588:
---
kkhatua commented on a change in pull request #1371: DRILL-6588: Make Sys
tables of nullable datatypes
URL: https://github.com/apache/drill/pull/1371#discussion_r201883141
##
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/BitToUserConnectionIterator.java
##
@@ -99,14 +102,18 @@ public void remove() {
public static class ConnectionInfo {
public String user;
+@Nonnull
Review comment:
Ok. Got it. This is part of the annotations-2.0.1 dependency that drill has
in its 3rd party library (`com.google.code.findbugs`). It might be used here in
a context different from what it was intended to be used for.
So, @arina-ielchiieva / @paul-rogers , should I go ahead with this usage or
revert to implementing the annotation class I originally had?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> System table columns incorrectly marked as non-nullable
>
>
> Key: DRILL-6588
> URL: https://issues.apache.org/jira/browse/DRILL-6588
> Project: Apache Drill
> Issue Type: Bug
> Components: Metadata
>Affects Versions: 1.13.0
>Reporter: Aman Sinha
>Assignee: Kunal Khatua
>Priority: Major
> Fix For: 1.14.0
>
>
> System table columns can contain null values but they are incorrectly marked
> as non-nullable as shown in example table below:
> {noformat}
> 0: jdbc:drill:drillbit=10.10.10.191> describe sys.boot;
> +---++--+
> | COLUMN_NAME | DATA_TYPE | IS_NULLABLE |
> +---++--+
> | name | CHARACTER VARYING | NO |
> | kind | CHARACTER VARYING | NO |
> | accessibleScopes | CHARACTER VARYING | NO |
> | optionScope | CHARACTER VARYING | NO |
> | status | CHARACTER VARYING | NO |
> | num_val | BIGINT | NO |
> | string_val | CHARACTER VARYING | NO |
> | bool_val | BOOLEAN | NO |
> | float_val | DOUBLE | NO |
> +---++--+{noformat}
>
> Note that several columns are nulls:
> {noformat}
> +---+--+--+-++-++--+---+
> | name | kind |
> accessibleScopes | optionScope | status | num_val | string_val | bool_val |
> float_val |
> +---+--+--+-++-++--+---+
> drill.exec.options.exec.udf.enable_dynamic_support | BOOLEAN | BOOT | BOOT |
> BOOT | null | null | true | null |{noformat}
>
> Because of the not-null metadata, the predicates on these tables such as
> `WHERE IS NULL` evaluate to FALSE which is incorrect.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6588) System table columns incorrectly marked as non-nullable
[
https://issues.apache.org/jira/browse/DRILL-6588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16540954#comment-16540954
]
ASF GitHub Bot commented on DRILL-6588:
---
kkhatua commented on a change in pull request #1371: DRILL-6588: Make Sys
tables of nullable datatypes
URL: https://github.com/apache/drill/pull/1371#discussion_r201882790
##
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/BitToUserConnectionIterator.java
##
@@ -99,14 +102,18 @@ public void remove() {
public static class ConnectionInfo {
public String user;
+@Nonnull
Review comment:
That is odd. I actually was referring to this:
https://blogs.oracle.com/java-platform-group/java-8s-new-type-annotations
I suspect the documentation is missing, because the
`javax.annotation.Nonnull` is meant for runtime code generation
```javax.annotation.Nonnull
@TypeQualifier
@Retention(value=RUNTIME)
@Documented
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> System table columns incorrectly marked as non-nullable
>
>
> Key: DRILL-6588
> URL: https://issues.apache.org/jira/browse/DRILL-6588
> Project: Apache Drill
> Issue Type: Bug
> Components: Metadata
>Affects Versions: 1.13.0
>Reporter: Aman Sinha
>Assignee: Kunal Khatua
>Priority: Major
> Fix For: 1.14.0
>
>
> System table columns can contain null values but they are incorrectly marked
> as non-nullable as shown in example table below:
> {noformat}
> 0: jdbc:drill:drillbit=10.10.10.191> describe sys.boot;
> +---++--+
> | COLUMN_NAME | DATA_TYPE | IS_NULLABLE |
> +---++--+
> | name | CHARACTER VARYING | NO |
> | kind | CHARACTER VARYING | NO |
> | accessibleScopes | CHARACTER VARYING | NO |
> | optionScope | CHARACTER VARYING | NO |
> | status | CHARACTER VARYING | NO |
> | num_val | BIGINT | NO |
> | string_val | CHARACTER VARYING | NO |
> | bool_val | BOOLEAN | NO |
> | float_val | DOUBLE | NO |
> +---++--+{noformat}
>
> Note that several columns are nulls:
> {noformat}
> +---+--+--+-++-++--+---+
> | name | kind |
> accessibleScopes | optionScope | status | num_val | string_val | bool_val |
> float_val |
> +---+--+--+-++-++--+---+
> drill.exec.options.exec.udf.enable_dynamic_support | BOOLEAN | BOOT | BOOT |
> BOOT | null | null | true | null |{noformat}
>
> Because of the not-null metadata, the predicates on these tables such as
> `WHERE IS NULL` evaluate to FALSE which is incorrect.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6588) System table columns incorrectly marked as non-nullable
[
https://issues.apache.org/jira/browse/DRILL-6588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16540951#comment-16540951
]
ASF GitHub Bot commented on DRILL-6588:
---
kkhatua commented on a change in pull request #1371: DRILL-6588: Make Sys
tables of nullable datatypes
URL: https://github.com/apache/drill/pull/1371#discussion_r201882044
##
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/store/RecordDataType.java
##
@@ -47,17 +48,22 @@
* @return the constructed type
*/
public final RelDataType getRowType(RelDataTypeFactory factory) {
-final List types = getFieldSqlTypeNames();
+final List> types =
getFieldSqlTypeNames();
final List names = getFieldNames();
final List fields = Lists.newArrayList();
-for (final SqlTypeName typeName : types) {
+
+for (ImmutablePair sqlTypePair : types) {
+ final SqlTypeName typeName = sqlTypePair.getLeft();
+ final RelDataType tmpRDT;
Review comment:
Will do
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> System table columns incorrectly marked as non-nullable
>
>
> Key: DRILL-6588
> URL: https://issues.apache.org/jira/browse/DRILL-6588
> Project: Apache Drill
> Issue Type: Bug
> Components: Metadata
>Affects Versions: 1.13.0
>Reporter: Aman Sinha
>Assignee: Kunal Khatua
>Priority: Major
> Fix For: 1.14.0
>
>
> System table columns can contain null values but they are incorrectly marked
> as non-nullable as shown in example table below:
> {noformat}
> 0: jdbc:drill:drillbit=10.10.10.191> describe sys.boot;
> +---++--+
> | COLUMN_NAME | DATA_TYPE | IS_NULLABLE |
> +---++--+
> | name | CHARACTER VARYING | NO |
> | kind | CHARACTER VARYING | NO |
> | accessibleScopes | CHARACTER VARYING | NO |
> | optionScope | CHARACTER VARYING | NO |
> | status | CHARACTER VARYING | NO |
> | num_val | BIGINT | NO |
> | string_val | CHARACTER VARYING | NO |
> | bool_val | BOOLEAN | NO |
> | float_val | DOUBLE | NO |
> +---++--+{noformat}
>
> Note that several columns are nulls:
> {noformat}
> +---+--+--+-++-++--+---+
> | name | kind |
> accessibleScopes | optionScope | status | num_val | string_val | bool_val |
> float_val |
> +---+--+--+-++-++--+---+
> drill.exec.options.exec.udf.enable_dynamic_support | BOOLEAN | BOOT | BOOT |
> BOOT | null | null | true | null |{noformat}
>
> Because of the not-null metadata, the predicates on these tables such as
> `WHERE IS NULL` evaluate to FALSE which is incorrect.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6588) System table columns incorrectly marked as non-nullable
[
https://issues.apache.org/jira/browse/DRILL-6588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16540950#comment-16540950
]
ASF GitHub Bot commented on DRILL-6588:
---
kkhatua commented on a change in pull request #1371: DRILL-6588: Make Sys
tables of nullable datatypes
URL: https://github.com/apache/drill/pull/1371#discussion_r201882038
##
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/store/pojo/PojoDataType.java
##
@@ -48,22 +51,25 @@ public PojoDataType(Class pojoClass) {
Class type = f.getType();
names.add(f.getName());
+ //Absence of annotation @Nonnull => (isNullable=true)
+ final boolean isNullable = f.isAnnotationPresent(Nonnull.class) ? false
: true;
Review comment:
Agreed. Will do this change
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> System table columns incorrectly marked as non-nullable
>
>
> Key: DRILL-6588
> URL: https://issues.apache.org/jira/browse/DRILL-6588
> Project: Apache Drill
> Issue Type: Bug
> Components: Metadata
>Affects Versions: 1.13.0
>Reporter: Aman Sinha
>Assignee: Kunal Khatua
>Priority: Major
> Fix For: 1.14.0
>
>
> System table columns can contain null values but they are incorrectly marked
> as non-nullable as shown in example table below:
> {noformat}
> 0: jdbc:drill:drillbit=10.10.10.191> describe sys.boot;
> +---++--+
> | COLUMN_NAME | DATA_TYPE | IS_NULLABLE |
> +---++--+
> | name | CHARACTER VARYING | NO |
> | kind | CHARACTER VARYING | NO |
> | accessibleScopes | CHARACTER VARYING | NO |
> | optionScope | CHARACTER VARYING | NO |
> | status | CHARACTER VARYING | NO |
> | num_val | BIGINT | NO |
> | string_val | CHARACTER VARYING | NO |
> | bool_val | BOOLEAN | NO |
> | float_val | DOUBLE | NO |
> +---++--+{noformat}
>
> Note that several columns are nulls:
> {noformat}
> +---+--+--+-++-++--+---+
> | name | kind |
> accessibleScopes | optionScope | status | num_val | string_val | bool_val |
> float_val |
> +---+--+--+-++-++--+---+
> drill.exec.options.exec.udf.enable_dynamic_support | BOOLEAN | BOOT | BOOT |
> BOOT | null | null | true | null |{noformat}
>
> Because of the not-null metadata, the predicates on these tables such as
> `WHERE IS NULL` evaluate to FALSE which is incorrect.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6588) System table columns incorrectly marked as non-nullable
[
https://issues.apache.org/jira/browse/DRILL-6588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16539804#comment-16539804
]
ASF GitHub Bot commented on DRILL-6588:
---
arina-ielchiieva commented on a change in pull request #1371: DRILL-6588: Make
Sys tables of nullable datatypes
URL: https://github.com/apache/drill/pull/1371#discussion_r201628034
##
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/BitToUserConnectionIterator.java
##
@@ -99,14 +102,18 @@ public void remove() {
public static class ConnectionInfo {
public String user;
+@Nonnull
Review comment:
The thing is that this is not native java annotation. It's absent in java 8
SE specification.
https://docs.oracle.com/javase/8/docs/api/javax/annotation/package-summary.html
http://mvnrepository.com/artifact/javax.annotation/javax.annotation-api/1.3.2
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> System table columns incorrectly marked as non-nullable
>
>
> Key: DRILL-6588
> URL: https://issues.apache.org/jira/browse/DRILL-6588
> Project: Apache Drill
> Issue Type: Bug
> Components: Metadata
>Affects Versions: 1.13.0
>Reporter: Aman Sinha
>Assignee: Kunal Khatua
>Priority: Major
> Fix For: 1.14.0
>
>
> System table columns can contain null values but they are incorrectly marked
> as non-nullable as shown in example table below:
> {noformat}
> 0: jdbc:drill:drillbit=10.10.10.191> describe sys.boot;
> +---++--+
> | COLUMN_NAME | DATA_TYPE | IS_NULLABLE |
> +---++--+
> | name | CHARACTER VARYING | NO |
> | kind | CHARACTER VARYING | NO |
> | accessibleScopes | CHARACTER VARYING | NO |
> | optionScope | CHARACTER VARYING | NO |
> | status | CHARACTER VARYING | NO |
> | num_val | BIGINT | NO |
> | string_val | CHARACTER VARYING | NO |
> | bool_val | BOOLEAN | NO |
> | float_val | DOUBLE | NO |
> +---++--+{noformat}
>
> Note that several columns are nulls:
> {noformat}
> +---+--+--+-++-++--+---+
> | name | kind |
> accessibleScopes | optionScope | status | num_val | string_val | bool_val |
> float_val |
> +---+--+--+-++-++--+---+
> drill.exec.options.exec.udf.enable_dynamic_support | BOOLEAN | BOOT | BOOT |
> BOOT | null | null | true | null |{noformat}
>
> Because of the not-null metadata, the predicates on these tables such as
> `WHERE IS NULL` evaluate to FALSE which is incorrect.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6588) System table columns incorrectly marked as non-nullable
[
https://issues.apache.org/jira/browse/DRILL-6588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16539802#comment-16539802
]
ASF GitHub Bot commented on DRILL-6588:
---
arina-ielchiieva commented on a change in pull request #1371: DRILL-6588: Make
Sys tables of nullable datatypes
URL: https://github.com/apache/drill/pull/1371#discussion_r201623807
##
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/store/RecordDataType.java
##
@@ -47,17 +48,22 @@
* @return the constructed type
*/
public final RelDataType getRowType(RelDataTypeFactory factory) {
-final List types = getFieldSqlTypeNames();
+final List> types =
getFieldSqlTypeNames();
Review comment:
I also agree with Paul, using Pair is not a good choice. Better to introduce
holder class.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> System table columns incorrectly marked as non-nullable
>
>
> Key: DRILL-6588
> URL: https://issues.apache.org/jira/browse/DRILL-6588
> Project: Apache Drill
> Issue Type: Bug
> Components: Metadata
>Affects Versions: 1.13.0
>Reporter: Aman Sinha
>Assignee: Kunal Khatua
>Priority: Major
> Fix For: 1.14.0
>
>
> System table columns can contain null values but they are incorrectly marked
> as non-nullable as shown in example table below:
> {noformat}
> 0: jdbc:drill:drillbit=10.10.10.191> describe sys.boot;
> +---++--+
> | COLUMN_NAME | DATA_TYPE | IS_NULLABLE |
> +---++--+
> | name | CHARACTER VARYING | NO |
> | kind | CHARACTER VARYING | NO |
> | accessibleScopes | CHARACTER VARYING | NO |
> | optionScope | CHARACTER VARYING | NO |
> | status | CHARACTER VARYING | NO |
> | num_val | BIGINT | NO |
> | string_val | CHARACTER VARYING | NO |
> | bool_val | BOOLEAN | NO |
> | float_val | DOUBLE | NO |
> +---++--+{noformat}
>
> Note that several columns are nulls:
> {noformat}
> +---+--+--+-++-++--+---+
> | name | kind |
> accessibleScopes | optionScope | status | num_val | string_val | bool_val |
> float_val |
> +---+--+--+-++-++--+---+
> drill.exec.options.exec.udf.enable_dynamic_support | BOOLEAN | BOOT | BOOT |
> BOOT | null | null | true | null |{noformat}
>
> Because of the not-null metadata, the predicates on these tables such as
> `WHERE IS NULL` evaluate to FALSE which is incorrect.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6588) System table columns incorrectly marked as non-nullable
[
https://issues.apache.org/jira/browse/DRILL-6588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16539803#comment-16539803
]
ASF GitHub Bot commented on DRILL-6588:
---
arina-ielchiieva commented on a change in pull request #1371: DRILL-6588: Make
Sys tables of nullable datatypes
URL: https://github.com/apache/drill/pull/1371#discussion_r201626723
##
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/store/pojo/PojoDataType.java
##
@@ -48,22 +51,25 @@ public PojoDataType(Class pojoClass) {
Class type = f.getType();
names.add(f.getName());
+ //Absence of annotation @Nonnull => (isNullable=true)
+ final boolean isNullable = f.isAnnotationPresent(Nonnull.class) ? false
: true;
Review comment:
This can be simplified to `! f.isAnnotationPresent(Nonnull.class)`.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> System table columns incorrectly marked as non-nullable
>
>
> Key: DRILL-6588
> URL: https://issues.apache.org/jira/browse/DRILL-6588
> Project: Apache Drill
> Issue Type: Bug
> Components: Metadata
>Affects Versions: 1.13.0
>Reporter: Aman Sinha
>Assignee: Kunal Khatua
>Priority: Major
> Fix For: 1.14.0
>
>
> System table columns can contain null values but they are incorrectly marked
> as non-nullable as shown in example table below:
> {noformat}
> 0: jdbc:drill:drillbit=10.10.10.191> describe sys.boot;
> +---++--+
> | COLUMN_NAME | DATA_TYPE | IS_NULLABLE |
> +---++--+
> | name | CHARACTER VARYING | NO |
> | kind | CHARACTER VARYING | NO |
> | accessibleScopes | CHARACTER VARYING | NO |
> | optionScope | CHARACTER VARYING | NO |
> | status | CHARACTER VARYING | NO |
> | num_val | BIGINT | NO |
> | string_val | CHARACTER VARYING | NO |
> | bool_val | BOOLEAN | NO |
> | float_val | DOUBLE | NO |
> +---++--+{noformat}
>
> Note that several columns are nulls:
> {noformat}
> +---+--+--+-++-++--+---+
> | name | kind |
> accessibleScopes | optionScope | status | num_val | string_val | bool_val |
> float_val |
> +---+--+--+-++-++--+---+
> drill.exec.options.exec.udf.enable_dynamic_support | BOOLEAN | BOOT | BOOT |
> BOOT | null | null | true | null |{noformat}
>
> Because of the not-null metadata, the predicates on these tables such as
> `WHERE IS NULL` evaluate to FALSE which is incorrect.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6588) System table columns incorrectly marked as non-nullable
[
https://issues.apache.org/jira/browse/DRILL-6588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16539801#comment-16539801
]
ASF GitHub Bot commented on DRILL-6588:
---
arina-ielchiieva commented on a change in pull request #1371: DRILL-6588: Make
Sys tables of nullable datatypes
URL: https://github.com/apache/drill/pull/1371#discussion_r201624058
##
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/store/RecordDataType.java
##
@@ -47,17 +48,22 @@
* @return the constructed type
*/
public final RelDataType getRowType(RelDataTypeFactory factory) {
-final List types = getFieldSqlTypeNames();
+final List> types =
getFieldSqlTypeNames();
final List names = getFieldNames();
final List fields = Lists.newArrayList();
-for (final SqlTypeName typeName : types) {
+
+for (ImmutablePair sqlTypePair : types) {
+ final SqlTypeName typeName = sqlTypePair.getLeft();
+ final RelDataType tmpRDT;
Review comment:
tmpRTD -> tmpDataType
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> System table columns incorrectly marked as non-nullable
>
>
> Key: DRILL-6588
> URL: https://issues.apache.org/jira/browse/DRILL-6588
> Project: Apache Drill
> Issue Type: Bug
> Components: Metadata
>Affects Versions: 1.13.0
>Reporter: Aman Sinha
>Assignee: Kunal Khatua
>Priority: Major
> Fix For: 1.14.0
>
>
> System table columns can contain null values but they are incorrectly marked
> as non-nullable as shown in example table below:
> {noformat}
> 0: jdbc:drill:drillbit=10.10.10.191> describe sys.boot;
> +---++--+
> | COLUMN_NAME | DATA_TYPE | IS_NULLABLE |
> +---++--+
> | name | CHARACTER VARYING | NO |
> | kind | CHARACTER VARYING | NO |
> | accessibleScopes | CHARACTER VARYING | NO |
> | optionScope | CHARACTER VARYING | NO |
> | status | CHARACTER VARYING | NO |
> | num_val | BIGINT | NO |
> | string_val | CHARACTER VARYING | NO |
> | bool_val | BOOLEAN | NO |
> | float_val | DOUBLE | NO |
> +---++--+{noformat}
>
> Note that several columns are nulls:
> {noformat}
> +---+--+--+-++-++--+---+
> | name | kind |
> accessibleScopes | optionScope | status | num_val | string_val | bool_val |
> float_val |
> +---+--+--+-++-++--+---+
> drill.exec.options.exec.udf.enable_dynamic_support | BOOLEAN | BOOT | BOOT |
> BOOT | null | null | true | null |{noformat}
>
> Because of the not-null metadata, the predicates on these tables such as
> `WHERE IS NULL` evaluate to FALSE which is incorrect.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6588) System table columns incorrectly marked as non-nullable
[
https://issues.apache.org/jira/browse/DRILL-6588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16539425#comment-16539425
]
ASF GitHub Bot commented on DRILL-6588:
---
kkhatua commented on a change in pull request #1371: DRILL-6588: Make Sys
tables of nullable datatypes
URL: https://github.com/apache/drill/pull/1371#discussion_r201539246
##
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/BitToUserConnectionIterator.java
##
@@ -99,14 +102,18 @@ public void remove() {
public static class ConnectionInfo {
public String user;
+@Nonnull
Review comment:
Thanks for pointing out the availability of these annotations. Didn't want
to reinvent the wheel. :)
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> System table columns incorrectly marked as non-nullable
>
>
> Key: DRILL-6588
> URL: https://issues.apache.org/jira/browse/DRILL-6588
> Project: Apache Drill
> Issue Type: Bug
> Components: Metadata
>Affects Versions: 1.13.0
>Reporter: Aman Sinha
>Assignee: Kunal Khatua
>Priority: Major
> Fix For: 1.14.0
>
>
> System table columns can contain null values but they are incorrectly marked
> as non-nullable as shown in example table below:
> {noformat}
> 0: jdbc:drill:drillbit=10.10.10.191> describe sys.boot;
> +---++--+
> | COLUMN_NAME | DATA_TYPE | IS_NULLABLE |
> +---++--+
> | name | CHARACTER VARYING | NO |
> | kind | CHARACTER VARYING | NO |
> | accessibleScopes | CHARACTER VARYING | NO |
> | optionScope | CHARACTER VARYING | NO |
> | status | CHARACTER VARYING | NO |
> | num_val | BIGINT | NO |
> | string_val | CHARACTER VARYING | NO |
> | bool_val | BOOLEAN | NO |
> | float_val | DOUBLE | NO |
> +---++--+{noformat}
>
> Note that several columns are nulls:
> {noformat}
> +---+--+--+-++-++--+---+
> | name | kind |
> accessibleScopes | optionScope | status | num_val | string_val | bool_val |
> float_val |
> +---+--+--+-++-++--+---+
> drill.exec.options.exec.udf.enable_dynamic_support | BOOLEAN | BOOT | BOOT |
> BOOT | null | null | true | null |{noformat}
>
> Because of the not-null metadata, the predicates on these tables such as
> `WHERE IS NULL` evaluate to FALSE which is incorrect.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6588) System table columns incorrectly marked as non-nullable
[
https://issues.apache.org/jira/browse/DRILL-6588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16539424#comment-16539424
]
ASF GitHub Bot commented on DRILL-6588:
---
kkhatua commented on a change in pull request #1371: DRILL-6588: Make Sys
tables of nullable datatypes
URL: https://github.com/apache/drill/pull/1371#discussion_r201539183
##
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/store/RecordDataType.java
##
@@ -47,17 +48,22 @@
* @return the constructed type
*/
public final RelDataType getRowType(RelDataTypeFactory factory) {
-final List types = getFieldSqlTypeNames();
+final List> types =
getFieldSqlTypeNames();
Review comment:
There are just 2 lists : `names` and `types`.
The third list (`fields`) is used as `factory.createStructType(fields,
names)`
I'm reluctant to change abstract methods because they might have usage
beyond the current SystemTable context.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> System table columns incorrectly marked as non-nullable
>
>
> Key: DRILL-6588
> URL: https://issues.apache.org/jira/browse/DRILL-6588
> Project: Apache Drill
> Issue Type: Bug
> Components: Metadata
>Affects Versions: 1.13.0
>Reporter: Aman Sinha
>Assignee: Kunal Khatua
>Priority: Major
> Fix For: 1.14.0
>
>
> System table columns can contain null values but they are incorrectly marked
> as non-nullable as shown in example table below:
> {noformat}
> 0: jdbc:drill:drillbit=10.10.10.191> describe sys.boot;
> +---++--+
> | COLUMN_NAME | DATA_TYPE | IS_NULLABLE |
> +---++--+
> | name | CHARACTER VARYING | NO |
> | kind | CHARACTER VARYING | NO |
> | accessibleScopes | CHARACTER VARYING | NO |
> | optionScope | CHARACTER VARYING | NO |
> | status | CHARACTER VARYING | NO |
> | num_val | BIGINT | NO |
> | string_val | CHARACTER VARYING | NO |
> | bool_val | BOOLEAN | NO |
> | float_val | DOUBLE | NO |
> +---++--+{noformat}
>
> Note that several columns are nulls:
> {noformat}
> +---+--+--+-++-++--+---+
> | name | kind |
> accessibleScopes | optionScope | status | num_val | string_val | bool_val |
> float_val |
> +---+--+--+-++-++--+---+
> drill.exec.options.exec.udf.enable_dynamic_support | BOOLEAN | BOOT | BOOT |
> BOOT | null | null | true | null |{noformat}
>
> Because of the not-null metadata, the predicates on these tables such as
> `WHERE IS NULL` evaluate to FALSE which is incorrect.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6588) System table columns incorrectly marked as non-nullable
[
https://issues.apache.org/jira/browse/DRILL-6588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16539401#comment-16539401
]
ASF GitHub Bot commented on DRILL-6588:
---
paul-rogers commented on a change in pull request #1371: DRILL-6588: Make Sys
tables of nullable datatypes
URL: https://github.com/apache/drill/pull/1371#discussion_r201533864
##
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/store/RecordDataType.java
##
@@ -47,17 +48,22 @@
* @return the constructed type
*/
public final RelDataType getRowType(RelDataTypeFactory factory) {
-final List types = getFieldSqlTypeNames();
+final List> types =
getFieldSqlTypeNames();
Review comment:
Better, but we still have three correlated lists. Can you combine the four
fields into a single struct (class) and have a single list?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> System table columns incorrectly marked as non-nullable
>
>
> Key: DRILL-6588
> URL: https://issues.apache.org/jira/browse/DRILL-6588
> Project: Apache Drill
> Issue Type: Bug
> Components: Metadata
>Affects Versions: 1.13.0
>Reporter: Aman Sinha
>Assignee: Kunal Khatua
>Priority: Major
> Fix For: 1.14.0
>
>
> System table columns can contain null values but they are incorrectly marked
> as non-nullable as shown in example table below:
> {noformat}
> 0: jdbc:drill:drillbit=10.10.10.191> describe sys.boot;
> +---++--+
> | COLUMN_NAME | DATA_TYPE | IS_NULLABLE |
> +---++--+
> | name | CHARACTER VARYING | NO |
> | kind | CHARACTER VARYING | NO |
> | accessibleScopes | CHARACTER VARYING | NO |
> | optionScope | CHARACTER VARYING | NO |
> | status | CHARACTER VARYING | NO |
> | num_val | BIGINT | NO |
> | string_val | CHARACTER VARYING | NO |
> | bool_val | BOOLEAN | NO |
> | float_val | DOUBLE | NO |
> +---++--+{noformat}
>
> Note that several columns are nulls:
> {noformat}
> +---+--+--+-++-++--+---+
> | name | kind |
> accessibleScopes | optionScope | status | num_val | string_val | bool_val |
> float_val |
> +---+--+--+-++-++--+---+
> drill.exec.options.exec.udf.enable_dynamic_support | BOOLEAN | BOOT | BOOT |
> BOOT | null | null | true | null |{noformat}
>
> Because of the not-null metadata, the predicates on these tables such as
> `WHERE IS NULL` evaluate to FALSE which is incorrect.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6588) System table columns incorrectly marked as non-nullable
[
https://issues.apache.org/jira/browse/DRILL-6588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16539400#comment-16539400
]
ASF GitHub Bot commented on DRILL-6588:
---
paul-rogers commented on a change in pull request #1371: DRILL-6588: Make Sys
tables of nullable datatypes
URL: https://github.com/apache/drill/pull/1371#discussion_r201533958
##
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/BitToUserConnectionIterator.java
##
@@ -99,14 +102,18 @@ public void remove() {
public static class ConnectionInfo {
public String user;
+@Nonnull
Review comment:
Nice use of the Java native annotation!
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> System table columns incorrectly marked as non-nullable
>
>
> Key: DRILL-6588
> URL: https://issues.apache.org/jira/browse/DRILL-6588
> Project: Apache Drill
> Issue Type: Bug
> Components: Metadata
>Affects Versions: 1.13.0
>Reporter: Aman Sinha
>Assignee: Kunal Khatua
>Priority: Major
> Fix For: 1.14.0
>
>
> System table columns can contain null values but they are incorrectly marked
> as non-nullable as shown in example table below:
> {noformat}
> 0: jdbc:drill:drillbit=10.10.10.191> describe sys.boot;
> +---++--+
> | COLUMN_NAME | DATA_TYPE | IS_NULLABLE |
> +---++--+
> | name | CHARACTER VARYING | NO |
> | kind | CHARACTER VARYING | NO |
> | accessibleScopes | CHARACTER VARYING | NO |
> | optionScope | CHARACTER VARYING | NO |
> | status | CHARACTER VARYING | NO |
> | num_val | BIGINT | NO |
> | string_val | CHARACTER VARYING | NO |
> | bool_val | BOOLEAN | NO |
> | float_val | DOUBLE | NO |
> +---++--+{noformat}
>
> Note that several columns are nulls:
> {noformat}
> +---+--+--+-++-++--+---+
> | name | kind |
> accessibleScopes | optionScope | status | num_val | string_val | bool_val |
> float_val |
> +---+--+--+-++-++--+---+
> drill.exec.options.exec.udf.enable_dynamic_support | BOOLEAN | BOOT | BOOT |
> BOOT | null | null | true | null |{noformat}
>
> Because of the not-null metadata, the predicates on these tables such as
> `WHERE IS NULL` evaluate to FALSE which is incorrect.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6588) System table columns incorrectly marked as non-nullable
[
https://issues.apache.org/jira/browse/DRILL-6588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16539399#comment-16539399
]
ASF GitHub Bot commented on DRILL-6588:
---
paul-rogers commented on a change in pull request #1371: DRILL-6588: Make Sys
tables of nullable datatypes
URL: https://github.com/apache/drill/pull/1371#discussion_r201533730
##
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/store/RecordDataType.java
##
@@ -49,15 +55,25 @@
public final RelDataType getRowType(RelDataTypeFactory factory) {
final List types = getFieldSqlTypeNames();
final List names = getFieldNames();
+final List nullables = getFieldNullability();
final List fields = Lists.newArrayList();
-for (final SqlTypeName typeName : types) {
+Iterator typesIter = types.listIterator();
+Iterator nullabilityIter = nullables.listIterator();
Review comment:
This is, in fact, the gist of the comment. While it is understandable to
want to leave he existing code as-is, adding more lists to the existing
parallel lists is not an improvement: it has caused a semi-wrong initial
implementation to become overly cumbersome.
If the implementation is local to this class or module, then leaving the
code cleaner after your change than when you found it is generally a good thing.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> System table columns incorrectly marked as non-nullable
>
>
> Key: DRILL-6588
> URL: https://issues.apache.org/jira/browse/DRILL-6588
> Project: Apache Drill
> Issue Type: Bug
> Components: Metadata
>Affects Versions: 1.13.0
>Reporter: Aman Sinha
>Assignee: Kunal Khatua
>Priority: Major
> Fix For: 1.14.0
>
>
> System table columns can contain null values but they are incorrectly marked
> as non-nullable as shown in example table below:
> {noformat}
> 0: jdbc:drill:drillbit=10.10.10.191> describe sys.boot;
> +---++--+
> | COLUMN_NAME | DATA_TYPE | IS_NULLABLE |
> +---++--+
> | name | CHARACTER VARYING | NO |
> | kind | CHARACTER VARYING | NO |
> | accessibleScopes | CHARACTER VARYING | NO |
> | optionScope | CHARACTER VARYING | NO |
> | status | CHARACTER VARYING | NO |
> | num_val | BIGINT | NO |
> | string_val | CHARACTER VARYING | NO |
> | bool_val | BOOLEAN | NO |
> | float_val | DOUBLE | NO |
> +---++--+{noformat}
>
> Note that several columns are nulls:
> {noformat}
> +---+--+--+-++-++--+---+
> | name | kind |
> accessibleScopes | optionScope | status | num_val | string_val | bool_val |
> float_val |
> +---+--+--+-++-++--+---+
> drill.exec.options.exec.udf.enable_dynamic_support | BOOLEAN | BOOT | BOOT |
> BOOT | null | null | true | null |{noformat}
>
> Because of the not-null metadata, the predicates on these tables such as
> `WHERE IS NULL` evaluate to FALSE which is incorrect.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6588) System table columns incorrectly marked as non-nullable
[
https://issues.apache.org/jira/browse/DRILL-6588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16539397#comment-16539397
]
ASF GitHub Bot commented on DRILL-6588:
---
kkhatua commented on issue #1371: DRILL-6588: Make Sys tables of nullable
datatypes
URL: https://github.com/apache/drill/pull/1371#issuecomment-404005272
@paul-rogers I've made the changes to use Java's Annotation class `Nonnull`
Please review the rest of the changes.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> System table columns incorrectly marked as non-nullable
>
>
> Key: DRILL-6588
> URL: https://issues.apache.org/jira/browse/DRILL-6588
> Project: Apache Drill
> Issue Type: Bug
> Components: Metadata
>Affects Versions: 1.13.0
>Reporter: Aman Sinha
>Assignee: Kunal Khatua
>Priority: Major
> Fix For: 1.14.0
>
>
> System table columns can contain null values but they are incorrectly marked
> as non-nullable as shown in example table below:
> {noformat}
> 0: jdbc:drill:drillbit=10.10.10.191> describe sys.boot;
> +---++--+
> | COLUMN_NAME | DATA_TYPE | IS_NULLABLE |
> +---++--+
> | name | CHARACTER VARYING | NO |
> | kind | CHARACTER VARYING | NO |
> | accessibleScopes | CHARACTER VARYING | NO |
> | optionScope | CHARACTER VARYING | NO |
> | status | CHARACTER VARYING | NO |
> | num_val | BIGINT | NO |
> | string_val | CHARACTER VARYING | NO |
> | bool_val | BOOLEAN | NO |
> | float_val | DOUBLE | NO |
> +---++--+{noformat}
>
> Note that several columns are nulls:
> {noformat}
> +---+--+--+-++-++--+---+
> | name | kind |
> accessibleScopes | optionScope | status | num_val | string_val | bool_val |
> float_val |
> +---+--+--+-++-++--+---+
> drill.exec.options.exec.udf.enable_dynamic_support | BOOLEAN | BOOT | BOOT |
> BOOT | null | null | true | null |{noformat}
>
> Because of the not-null metadata, the predicates on these tables such as
> `WHERE IS NULL` evaluate to FALSE which is incorrect.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6588) System table columns incorrectly marked as non-nullable
[
https://issues.apache.org/jira/browse/DRILL-6588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16539372#comment-16539372
]
ASF GitHub Bot commented on DRILL-6588:
---
kkhatua commented on a change in pull request #1371: DRILL-6588: Make Sys
tables of nullable datatypes
URL: https://github.com/apache/drill/pull/1371#discussion_r201527711
##
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/store/pojo/PojoDataType.java
##
@@ -36,6 +36,7 @@
private final List types = Lists.newArrayList();
private final List names = Lists.newArrayList();
+ private final List nullables = Lists.newArrayList();
Review comment:
RecordDataType uses the `names` and `types` lists separately. So, I'll need
atleast 2 lists.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> System table columns incorrectly marked as non-nullable
>
>
> Key: DRILL-6588
> URL: https://issues.apache.org/jira/browse/DRILL-6588
> Project: Apache Drill
> Issue Type: Bug
> Components: Metadata
>Affects Versions: 1.13.0
>Reporter: Aman Sinha
>Assignee: Kunal Khatua
>Priority: Major
> Fix For: 1.14.0
>
>
> System table columns can contain null values but they are incorrectly marked
> as non-nullable as shown in example table below:
> {noformat}
> 0: jdbc:drill:drillbit=10.10.10.191> describe sys.boot;
> +---++--+
> | COLUMN_NAME | DATA_TYPE | IS_NULLABLE |
> +---++--+
> | name | CHARACTER VARYING | NO |
> | kind | CHARACTER VARYING | NO |
> | accessibleScopes | CHARACTER VARYING | NO |
> | optionScope | CHARACTER VARYING | NO |
> | status | CHARACTER VARYING | NO |
> | num_val | BIGINT | NO |
> | string_val | CHARACTER VARYING | NO |
> | bool_val | BOOLEAN | NO |
> | float_val | DOUBLE | NO |
> +---++--+{noformat}
>
> Note that several columns are nulls:
> {noformat}
> +---+--+--+-++-++--+---+
> | name | kind |
> accessibleScopes | optionScope | status | num_val | string_val | bool_val |
> float_val |
> +---+--+--+-++-++--+---+
> drill.exec.options.exec.udf.enable_dynamic_support | BOOLEAN | BOOT | BOOT |
> BOOT | null | null | true | null |{noformat}
>
> Because of the not-null metadata, the predicates on these tables such as
> `WHERE IS NULL` evaluate to FALSE which is incorrect.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6588) System table columns incorrectly marked as non-nullable
[
https://issues.apache.org/jira/browse/DRILL-6588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16539371#comment-16539371
]
ASF GitHub Bot commented on DRILL-6588:
---
kkhatua commented on a change in pull request #1371: DRILL-6588: Make Sys
tables of nullable datatypes
URL: https://github.com/apache/drill/pull/1371#discussion_r201527575
##
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/store/pojo/Nullability.java
##
@@ -0,0 +1,32 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.drill.exec.store.pojo;
+
+import java.lang.annotation.ElementType;
+import java.lang.annotation.Retention;
+import java.lang.annotation.RetentionPolicy;
+import java.lang.annotation.Target;
+
+/**
+ * Indicates Nullability
+ */
+@Retention(RetentionPolicy.RUNTIME)
+@Target(ElementType.FIELD)
+public @interface Nullability {
Review comment:
That would help. Let me take a look.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> System table columns incorrectly marked as non-nullable
>
>
> Key: DRILL-6588
> URL: https://issues.apache.org/jira/browse/DRILL-6588
> Project: Apache Drill
> Issue Type: Bug
> Components: Metadata
>Affects Versions: 1.13.0
>Reporter: Aman Sinha
>Assignee: Kunal Khatua
>Priority: Major
> Fix For: 1.14.0
>
>
> System table columns can contain null values but they are incorrectly marked
> as non-nullable as shown in example table below:
> {noformat}
> 0: jdbc:drill:drillbit=10.10.10.191> describe sys.boot;
> +---++--+
> | COLUMN_NAME | DATA_TYPE | IS_NULLABLE |
> +---++--+
> | name | CHARACTER VARYING | NO |
> | kind | CHARACTER VARYING | NO |
> | accessibleScopes | CHARACTER VARYING | NO |
> | optionScope | CHARACTER VARYING | NO |
> | status | CHARACTER VARYING | NO |
> | num_val | BIGINT | NO |
> | string_val | CHARACTER VARYING | NO |
> | bool_val | BOOLEAN | NO |
> | float_val | DOUBLE | NO |
> +---++--+{noformat}
>
> Note that several columns are nulls:
> {noformat}
> +---+--+--+-++-++--+---+
> | name | kind |
> accessibleScopes | optionScope | status | num_val | string_val | bool_val |
> float_val |
> +---+--+--+-++-++--+---+
> drill.exec.options.exec.udf.enable_dynamic_support | BOOLEAN | BOOT | BOOT |
> BOOT | null | null | true | null |{noformat}
>
> Because of the not-null metadata, the predicates on these tables such as
> `WHERE IS NULL` evaluate to FALSE which is incorrect.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6588) System table columns incorrectly marked as non-nullable
[
https://issues.apache.org/jira/browse/DRILL-6588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16539368#comment-16539368
]
ASF GitHub Bot commented on DRILL-6588:
---
kkhatua commented on a change in pull request #1371: DRILL-6588: Make Sys
tables of nullable datatypes
URL: https://github.com/apache/drill/pull/1371#discussion_r201527356
##
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/store/pojo/PojoDataType.java
##
@@ -48,6 +49,11 @@ public PojoDataType(Class pojoClass) {
Class type = f.getType();
names.add(f.getName());
+ //Look up @Nullability for nullable property
+ Nullability nullability = f.getDeclaredAnnotation(Nullability.class);
+ nullables.add(nullability == null ? //Absence of annotation =>
(isNullable=true)
Review comment:
It seemed very verbose, but that makes sense. I'll make this change.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> System table columns incorrectly marked as non-nullable
>
>
> Key: DRILL-6588
> URL: https://issues.apache.org/jira/browse/DRILL-6588
> Project: Apache Drill
> Issue Type: Bug
> Components: Metadata
>Affects Versions: 1.13.0
>Reporter: Aman Sinha
>Assignee: Kunal Khatua
>Priority: Major
> Fix For: 1.14.0
>
>
> System table columns can contain null values but they are incorrectly marked
> as non-nullable as shown in example table below:
> {noformat}
> 0: jdbc:drill:drillbit=10.10.10.191> describe sys.boot;
> +---++--+
> | COLUMN_NAME | DATA_TYPE | IS_NULLABLE |
> +---++--+
> | name | CHARACTER VARYING | NO |
> | kind | CHARACTER VARYING | NO |
> | accessibleScopes | CHARACTER VARYING | NO |
> | optionScope | CHARACTER VARYING | NO |
> | status | CHARACTER VARYING | NO |
> | num_val | BIGINT | NO |
> | string_val | CHARACTER VARYING | NO |
> | bool_val | BOOLEAN | NO |
> | float_val | DOUBLE | NO |
> +---++--+{noformat}
>
> Note that several columns are nulls:
> {noformat}
> +---+--+--+-++-++--+---+
> | name | kind |
> accessibleScopes | optionScope | status | num_val | string_val | bool_val |
> float_val |
> +---+--+--+-++-++--+---+
> drill.exec.options.exec.udf.enable_dynamic_support | BOOLEAN | BOOT | BOOT |
> BOOT | null | null | true | null |{noformat}
>
> Because of the not-null metadata, the predicates on these tables such as
> `WHERE IS NULL` evaluate to FALSE which is incorrect.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6588) System table columns incorrectly marked as non-nullable
[
https://issues.apache.org/jira/browse/DRILL-6588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16539366#comment-16539366
]
ASF GitHub Bot commented on DRILL-6588:
---
kkhatua commented on a change in pull request #1371: DRILL-6588: Make Sys
tables of nullable datatypes
URL: https://github.com/apache/drill/pull/1371#discussion_r201527129
##
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/store/RecordDataType.java
##
@@ -49,15 +55,25 @@
public final RelDataType getRowType(RelDataTypeFactory factory) {
final List types = getFieldSqlTypeNames();
final List names = getFieldNames();
+final List nullables = getFieldNullability();
final List fields = Lists.newArrayList();
-for (final SqlTypeName typeName : types) {
+Iterator typesIter = types.listIterator();
+Iterator nullabilityIter = nullables.listIterator();
Review comment:
I wanted to avoid changing existing implementation, so I added an extra list
and iterated in parallel.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> System table columns incorrectly marked as non-nullable
>
>
> Key: DRILL-6588
> URL: https://issues.apache.org/jira/browse/DRILL-6588
> Project: Apache Drill
> Issue Type: Bug
> Components: Metadata
>Affects Versions: 1.13.0
>Reporter: Aman Sinha
>Assignee: Kunal Khatua
>Priority: Major
> Fix For: 1.14.0
>
>
> System table columns can contain null values but they are incorrectly marked
> as non-nullable as shown in example table below:
> {noformat}
> 0: jdbc:drill:drillbit=10.10.10.191> describe sys.boot;
> +---++--+
> | COLUMN_NAME | DATA_TYPE | IS_NULLABLE |
> +---++--+
> | name | CHARACTER VARYING | NO |
> | kind | CHARACTER VARYING | NO |
> | accessibleScopes | CHARACTER VARYING | NO |
> | optionScope | CHARACTER VARYING | NO |
> | status | CHARACTER VARYING | NO |
> | num_val | BIGINT | NO |
> | string_val | CHARACTER VARYING | NO |
> | bool_val | BOOLEAN | NO |
> | float_val | DOUBLE | NO |
> +---++--+{noformat}
>
> Note that several columns are nulls:
> {noformat}
> +---+--+--+-++-++--+---+
> | name | kind |
> accessibleScopes | optionScope | status | num_val | string_val | bool_val |
> float_val |
> +---+--+--+-++-++--+---+
> drill.exec.options.exec.udf.enable_dynamic_support | BOOLEAN | BOOT | BOOT |
> BOOT | null | null | true | null |{noformat}
>
> Because of the not-null metadata, the predicates on these tables such as
> `WHERE IS NULL` evaluate to FALSE which is incorrect.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6588) System table columns incorrectly marked as non-nullable
[
https://issues.apache.org/jira/browse/DRILL-6588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16539343#comment-16539343
]
ASF GitHub Bot commented on DRILL-6588:
---
paul-rogers commented on a change in pull request #1371: DRILL-6588: Make Sys
tables of nullable datatypes
URL: https://github.com/apache/drill/pull/1371#discussion_r201523513
##
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/store/pojo/PojoDataType.java
##
@@ -48,6 +49,11 @@ public PojoDataType(Class pojoClass) {
Class type = f.getType();
names.add(f.getName());
+ //Look up @Nullability for nullable property
+ Nullability nullability = f.getDeclaredAnnotation(Nullability.class);
+ nullables.add(nullability == null ? //Absence of annotation =>
(isNullable=true)
Review comment:
`f.isAnnotationPresent(Nullability.class)` returns a `boolean` result; may
simplify the code here
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> System table columns incorrectly marked as non-nullable
>
>
> Key: DRILL-6588
> URL: https://issues.apache.org/jira/browse/DRILL-6588
> Project: Apache Drill
> Issue Type: Bug
> Components: Metadata
>Affects Versions: 1.13.0
>Reporter: Aman Sinha
>Assignee: Kunal Khatua
>Priority: Major
> Fix For: 1.14.0
>
>
> System table columns can contain null values but they are incorrectly marked
> as non-nullable as shown in example table below:
> {noformat}
> 0: jdbc:drill:drillbit=10.10.10.191> describe sys.boot;
> +---++--+
> | COLUMN_NAME | DATA_TYPE | IS_NULLABLE |
> +---++--+
> | name | CHARACTER VARYING | NO |
> | kind | CHARACTER VARYING | NO |
> | accessibleScopes | CHARACTER VARYING | NO |
> | optionScope | CHARACTER VARYING | NO |
> | status | CHARACTER VARYING | NO |
> | num_val | BIGINT | NO |
> | string_val | CHARACTER VARYING | NO |
> | bool_val | BOOLEAN | NO |
> | float_val | DOUBLE | NO |
> +---++--+{noformat}
>
> Note that several columns are nulls:
> {noformat}
> +---+--+--+-++-++--+---+
> | name | kind |
> accessibleScopes | optionScope | status | num_val | string_val | bool_val |
> float_val |
> +---+--+--+-++-++--+---+
> drill.exec.options.exec.udf.enable_dynamic_support | BOOLEAN | BOOT | BOOT |
> BOOT | null | null | true | null |{noformat}
>
> Because of the not-null metadata, the predicates on these tables such as
> `WHERE IS NULL` evaluate to FALSE which is incorrect.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6588) System table columns incorrectly marked as non-nullable
[
https://issues.apache.org/jira/browse/DRILL-6588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16539344#comment-16539344
]
ASF GitHub Bot commented on DRILL-6588:
---
paul-rogers commented on a change in pull request #1371: DRILL-6588: Make Sys
tables of nullable datatypes
URL: https://github.com/apache/drill/pull/1371#discussion_r201523626
##
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/store/pojo/PojoDataType.java
##
@@ -84,4 +90,8 @@ public PojoDataType(Class pojoClass) {
return names;
}
+ @Override
+ public List getFieldNullability() {
Review comment:
Same comment about the awkwardness of correlated lists...
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> System table columns incorrectly marked as non-nullable
>
>
> Key: DRILL-6588
> URL: https://issues.apache.org/jira/browse/DRILL-6588
> Project: Apache Drill
> Issue Type: Bug
> Components: Metadata
>Affects Versions: 1.13.0
>Reporter: Aman Sinha
>Assignee: Kunal Khatua
>Priority: Major
> Fix For: 1.14.0
>
>
> System table columns can contain null values but they are incorrectly marked
> as non-nullable as shown in example table below:
> {noformat}
> 0: jdbc:drill:drillbit=10.10.10.191> describe sys.boot;
> +---++--+
> | COLUMN_NAME | DATA_TYPE | IS_NULLABLE |
> +---++--+
> | name | CHARACTER VARYING | NO |
> | kind | CHARACTER VARYING | NO |
> | accessibleScopes | CHARACTER VARYING | NO |
> | optionScope | CHARACTER VARYING | NO |
> | status | CHARACTER VARYING | NO |
> | num_val | BIGINT | NO |
> | string_val | CHARACTER VARYING | NO |
> | bool_val | BOOLEAN | NO |
> | float_val | DOUBLE | NO |
> +---++--+{noformat}
>
> Note that several columns are nulls:
> {noformat}
> +---+--+--+-++-++--+---+
> | name | kind |
> accessibleScopes | optionScope | status | num_val | string_val | bool_val |
> float_val |
> +---+--+--+-++-++--+---+
> drill.exec.options.exec.udf.enable_dynamic_support | BOOLEAN | BOOT | BOOT |
> BOOT | null | null | true | null |{noformat}
>
> Because of the not-null metadata, the predicates on these tables such as
> `WHERE IS NULL` evaluate to FALSE which is incorrect.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6588) System table columns incorrectly marked as non-nullable
[
https://issues.apache.org/jira/browse/DRILL-6588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16539340#comment-16539340
]
ASF GitHub Bot commented on DRILL-6588:
---
paul-rogers commented on a change in pull request #1371: DRILL-6588: Make Sys
tables of nullable datatypes
URL: https://github.com/apache/drill/pull/1371#discussion_r201522572
##
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/store/RecordDataType.java
##
@@ -49,15 +55,25 @@
public final RelDataType getRowType(RelDataTypeFactory factory) {
final List types = getFieldSqlTypeNames();
final List names = getFieldNames();
+final List nullables = getFieldNullability();
final List fields = Lists.newArrayList();
-for (final SqlTypeName typeName : types) {
+Iterator typesIter = types.listIterator();
+Iterator nullabilityIter = nullables.listIterator();
Review comment:
We now four correlated lists. This is often difficult to reason about and
maintain.
Is it possible to have a single list of, say, `FieldDefn` objects, each of
which has a name, type, nullable and `RelDataType`?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> System table columns incorrectly marked as non-nullable
>
>
> Key: DRILL-6588
> URL: https://issues.apache.org/jira/browse/DRILL-6588
> Project: Apache Drill
> Issue Type: Bug
> Components: Metadata
>Affects Versions: 1.13.0
>Reporter: Aman Sinha
>Assignee: Kunal Khatua
>Priority: Major
> Fix For: 1.14.0
>
>
> System table columns can contain null values but they are incorrectly marked
> as non-nullable as shown in example table below:
> {noformat}
> 0: jdbc:drill:drillbit=10.10.10.191> describe sys.boot;
> +---++--+
> | COLUMN_NAME | DATA_TYPE | IS_NULLABLE |
> +---++--+
> | name | CHARACTER VARYING | NO |
> | kind | CHARACTER VARYING | NO |
> | accessibleScopes | CHARACTER VARYING | NO |
> | optionScope | CHARACTER VARYING | NO |
> | status | CHARACTER VARYING | NO |
> | num_val | BIGINT | NO |
> | string_val | CHARACTER VARYING | NO |
> | bool_val | BOOLEAN | NO |
> | float_val | DOUBLE | NO |
> +---++--+{noformat}
>
> Note that several columns are nulls:
> {noformat}
> +---+--+--+-++-++--+---+
> | name | kind |
> accessibleScopes | optionScope | status | num_val | string_val | bool_val |
> float_val |
> +---+--+--+-++-++--+---+
> drill.exec.options.exec.udf.enable_dynamic_support | BOOLEAN | BOOT | BOOT |
> BOOT | null | null | true | null |{noformat}
>
> Because of the not-null metadata, the predicates on these tables such as
> `WHERE IS NULL` evaluate to FALSE which is incorrect.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6588) System table columns incorrectly marked as non-nullable
[
https://issues.apache.org/jira/browse/DRILL-6588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16539342#comment-16539342
]
ASF GitHub Bot commented on DRILL-6588:
---
paul-rogers commented on a change in pull request #1371: DRILL-6588: Make Sys
tables of nullable datatypes
URL: https://github.com/apache/drill/pull/1371#discussion_r201523179
##
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/store/pojo/PojoDataType.java
##
@@ -36,6 +36,7 @@
private final List types = Lists.newArrayList();
private final List names = Lists.newArrayList();
+ private final List nullables = Lists.newArrayList();
Review comment:
Again, single list of objects rather than three correlated list of
primitives?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> System table columns incorrectly marked as non-nullable
>
>
> Key: DRILL-6588
> URL: https://issues.apache.org/jira/browse/DRILL-6588
> Project: Apache Drill
> Issue Type: Bug
> Components: Metadata
>Affects Versions: 1.13.0
>Reporter: Aman Sinha
>Assignee: Kunal Khatua
>Priority: Major
> Fix For: 1.14.0
>
>
> System table columns can contain null values but they are incorrectly marked
> as non-nullable as shown in example table below:
> {noformat}
> 0: jdbc:drill:drillbit=10.10.10.191> describe sys.boot;
> +---++--+
> | COLUMN_NAME | DATA_TYPE | IS_NULLABLE |
> +---++--+
> | name | CHARACTER VARYING | NO |
> | kind | CHARACTER VARYING | NO |
> | accessibleScopes | CHARACTER VARYING | NO |
> | optionScope | CHARACTER VARYING | NO |
> | status | CHARACTER VARYING | NO |
> | num_val | BIGINT | NO |
> | string_val | CHARACTER VARYING | NO |
> | bool_val | BOOLEAN | NO |
> | float_val | DOUBLE | NO |
> +---++--+{noformat}
>
> Note that several columns are nulls:
> {noformat}
> +---+--+--+-++-++--+---+
> | name | kind |
> accessibleScopes | optionScope | status | num_val | string_val | bool_val |
> float_val |
> +---+--+--+-++-++--+---+
> drill.exec.options.exec.udf.enable_dynamic_support | BOOLEAN | BOOT | BOOT |
> BOOT | null | null | true | null |{noformat}
>
> Because of the not-null metadata, the predicates on these tables such as
> `WHERE IS NULL` evaluate to FALSE which is incorrect.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6588) System table columns incorrectly marked as non-nullable
[
https://issues.apache.org/jira/browse/DRILL-6588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16539341#comment-16539341
]
ASF GitHub Bot commented on DRILL-6588:
---
paul-rogers commented on a change in pull request #1371: DRILL-6588: Make Sys
tables of nullable datatypes
URL: https://github.com/apache/drill/pull/1371#discussion_r201523061
##
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/store/pojo/Nullability.java
##
@@ -0,0 +1,32 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.drill.exec.store.pojo;
+
+import java.lang.annotation.ElementType;
+import java.lang.annotation.Retention;
+import java.lang.annotation.RetentionPolicy;
+import java.lang.annotation.Target;
+
+/**
+ * Indicates Nullability
+ */
+@Retention(RetentionPolicy.RUNTIME)
+@Target(ElementType.FIELD)
+public @interface Nullability {
Review comment:
Turns out that Java has a existing `@Nonnull` and `@Nullable` annotations in
`java.annotation`. Can we use those?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> System table columns incorrectly marked as non-nullable
>
>
> Key: DRILL-6588
> URL: https://issues.apache.org/jira/browse/DRILL-6588
> Project: Apache Drill
> Issue Type: Bug
> Components: Metadata
>Affects Versions: 1.13.0
>Reporter: Aman Sinha
>Assignee: Kunal Khatua
>Priority: Major
> Fix For: 1.14.0
>
>
> System table columns can contain null values but they are incorrectly marked
> as non-nullable as shown in example table below:
> {noformat}
> 0: jdbc:drill:drillbit=10.10.10.191> describe sys.boot;
> +---++--+
> | COLUMN_NAME | DATA_TYPE | IS_NULLABLE |
> +---++--+
> | name | CHARACTER VARYING | NO |
> | kind | CHARACTER VARYING | NO |
> | accessibleScopes | CHARACTER VARYING | NO |
> | optionScope | CHARACTER VARYING | NO |
> | status | CHARACTER VARYING | NO |
> | num_val | BIGINT | NO |
> | string_val | CHARACTER VARYING | NO |
> | bool_val | BOOLEAN | NO |
> | float_val | DOUBLE | NO |
> +---++--+{noformat}
>
> Note that several columns are nulls:
> {noformat}
> +---+--+--+-++-++--+---+
> | name | kind |
> accessibleScopes | optionScope | status | num_val | string_val | bool_val |
> float_val |
> +---+--+--+-++-++--+---+
> drill.exec.options.exec.udf.enable_dynamic_support | BOOLEAN | BOOT | BOOT |
> BOOT | null | null | true | null |{noformat}
>
> Because of the not-null metadata, the predicates on these tables such as
> `WHERE IS NULL` evaluate to FALSE which is incorrect.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6588) System table columns incorrectly marked as non-nullable
[ https://issues.apache.org/jira/browse/DRILL-6588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16539040#comment-16539040 ] ASF GitHub Bot commented on DRILL-6588: --- kkhatua opened a new pull request #1371: DRILL-6588: Make Sys tables of nullable datatypes URL: https://github.com/apache/drill/pull/1371 This addresses the issue of columns in the System tables being marked as non-nullable. While these tables are immutable, they can carry nullable values as well. (e.g. `num_val` in `sys.options`) This commit introduces an annotation for the `PojoReader` that applies the nullable property if explicitly defined for a column (i.e. explicitly defined `isNullable` for a member of the POJO instance). ``` apache drill 1.14.0-SNAPSHOT "just drill it" 0: jdbc:drill:schema=sys> select count(*) from sys.options where num_val is null; +-+ | EXPR$0 | +-+ | 108 | +-+ 1 row selected (2.703 seconds) 0: jdbc:drill:schema=sys> select count(*) from sys.options where isnull(num_val); +-+ | EXPR$0 | +-+ | 108 | +-+ 1 row selected (0.3 seconds) 0: jdbc:drill:schema=sys> select distinct is_nullable, count(*) from INFORMATION_SCHEMA.`COLUMNS` where table_schema = 'sys' group by is_nullable; +--+-+ | is_nullable | EXPR$1 | +--+-+ | NO | 36 | | YES | 50 | +--+-+ 2 rows selected (0.69 seconds) 0: jdbc:drill:schema=sys> describe options; +---++--+ |COLUMN_NAME| DATA_TYPE | IS_NULLABLE | +---++--+ | name | CHARACTER VARYING | NO | | kind | CHARACTER VARYING | NO | | accessibleScopes | CHARACTER VARYING | NO | | optionScope | CHARACTER VARYING | NO | | status| CHARACTER VARYING | NO | | num_val | BIGINT | YES | | string_val| CHARACTER VARYING | YES | | bool_val | BOOLEAN| YES | | float_val | DOUBLE | YES | +---++--+ 9 rows selected (0.221 seconds) 0: jdbc:drill:schema=sys> describe internal_options; +---++--+ |COLUMN_NAME| DATA_TYPE | IS_NULLABLE | +---++--+ | name | CHARACTER VARYING | NO | | kind | CHARACTER VARYING | NO | | accessibleScopes | CHARACTER VARYING | NO | | optionScope | CHARACTER VARYING | NO | | status| CHARACTER VARYING | NO | | num_val | BIGINT | YES | | string_val| CHARACTER VARYING | YES | | bool_val | BOOLEAN| YES | | float_val | DOUBLE | YES | +---++--+ 9 rows selected (0.185 seconds) 0: jdbc:drill:schema=sys> describe options_val; +---++--+ |COLUMN_NAME| DATA_TYPE | IS_NULLABLE | +---++--+ | name | CHARACTER VARYING | NO | | kind | CHARACTER VARYING | NO | | accessibleScopes | CHARACTER VARYING | NO | | val | CHARACTER VARYING | YES | | optionScope | CHARACTER VARYING | NO | +---++--+ 5 rows selected (0.183 seconds) 0: jdbc:drill:schema=sys> describe profiles_json; +--++--+ | COLUMN_NAME | DATA_TYPE | IS_NULLABLE | +--++--+ | queryId | CHARACTER VARYING | NO | | json | CHARACTER VARYING | YES | +--++--+ 2 rows selected (0.727 seconds) 0: jdbc:drill:schema=sys> describe profiles; +--++--+ | COLUMN_NAME | DATA_TYPE | IS_NULLABLE | +--++--+ | queryId | CHARACTER VARYING | NO | | startTime| TIMESTAMP | YES | | foreman | CHARACTER VARYING | NO | | fragments| BIGINT | YES | | user | CHARACTER VARYING | YES | | queue| CHARACTER VARYING | YES | | planTime | BIGINT
[jira] [Commented] (DRILL-6588) System table columns incorrectly marked as non-nullable
[
https://issues.apache.org/jira/browse/DRILL-6588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16539042#comment-16539042
]
ASF GitHub Bot commented on DRILL-6588:
---
kkhatua commented on issue #1371: DRILL-6588: Make Sys tables of nullable
datatypes
URL: https://github.com/apache/drill/pull/1371#issuecomment-403914728
@amansinha100 can you review this?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> System table columns incorrectly marked as non-nullable
>
>
> Key: DRILL-6588
> URL: https://issues.apache.org/jira/browse/DRILL-6588
> Project: Apache Drill
> Issue Type: Bug
> Components: Metadata
>Affects Versions: 1.13.0
>Reporter: Aman Sinha
>Assignee: Kunal Khatua
>Priority: Major
> Fix For: 1.14.0
>
>
> System table columns can contain null values but they are incorrectly marked
> as non-nullable as shown in example table below:
> {noformat}
> 0: jdbc:drill:drillbit=10.10.10.191> describe sys.boot;
> +---++--+
> | COLUMN_NAME | DATA_TYPE | IS_NULLABLE |
> +---++--+
> | name | CHARACTER VARYING | NO |
> | kind | CHARACTER VARYING | NO |
> | accessibleScopes | CHARACTER VARYING | NO |
> | optionScope | CHARACTER VARYING | NO |
> | status | CHARACTER VARYING | NO |
> | num_val | BIGINT | NO |
> | string_val | CHARACTER VARYING | NO |
> | bool_val | BOOLEAN | NO |
> | float_val | DOUBLE | NO |
> +---++--+{noformat}
>
> Note that several columns are nulls:
> {noformat}
> +---+--+--+-++-++--+---+
> | name | kind |
> accessibleScopes | optionScope | status | num_val | string_val | bool_val |
> float_val |
> +---+--+--+-++-++--+---+
> drill.exec.options.exec.udf.enable_dynamic_support | BOOLEAN | BOOT | BOOT |
> BOOT | null | null | true | null |{noformat}
>
> Because of the not-null metadata, the predicates on these tables such as
> `WHERE IS NULL` evaluate to FALSE which is incorrect.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
