[
https://issues.apache.org/jira/browse/DRILL-6588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16539040#comment-16539040
]
ASF GitHub Bot commented on DRILL-6588:
---------------------------------------
kkhatua opened a new pull request #1371: DRILL-6588: Make Sys tables of
nullable datatypes
URL: https://github.com/apache/drill/pull/1371
This addresses the issue of columns in the System tables being marked as
non-nullable. While these tables are immutable, they can carry nullable values
as well. (e.g. `num_val` in `sys.options`)
This commit introduces an annotation for the `PojoReader` that applies the
nullable property if explicitly defined for a column (i.e. explicitly defined
`isNullable` for a member of the POJO instance).
```
apache drill 1.14.0-SNAPSHOT
"just drill it"
0: jdbc:drill:schema=sys> select count(*) from sys.options where num_val is
null;
+---------+
| EXPR$0 |
+---------+
| 108 |
+---------+
1 row selected (2.703 seconds)
0: jdbc:drill:schema=sys> select count(*) from sys.options where
isnull(num_val);
+---------+
| EXPR$0 |
+---------+
| 108 |
+---------+
1 row selected (0.3 seconds)
0: jdbc:drill:schema=sys> select distinct is_nullable, count(*) from
INFORMATION_SCHEMA.`COLUMNS` where table_schema = 'sys' group by is_nullable;
+--------------+---------+
| is_nullable | EXPR$1 |
+--------------+---------+
| NO | 36 |
| YES | 50 |
+--------------+---------+
2 rows selected (0.69 seconds)
0: jdbc:drill:schema=sys> describe options;
+-------------------+--------------------+--------------+
| COLUMN_NAME | DATA_TYPE | IS_NULLABLE |
+-------------------+--------------------+--------------+
| name | CHARACTER VARYING | NO |
| kind | CHARACTER VARYING | NO |
| accessibleScopes | CHARACTER VARYING | NO |
| optionScope | CHARACTER VARYING | NO |
| status | CHARACTER VARYING | NO |
| num_val | BIGINT | YES |
| string_val | CHARACTER VARYING | YES |
| bool_val | BOOLEAN | YES |
| float_val | DOUBLE | YES |
+-------------------+--------------------+--------------+
9 rows selected (0.221 seconds)
0: jdbc:drill:schema=sys> describe internal_options;
+-------------------+--------------------+--------------+
| COLUMN_NAME | DATA_TYPE | IS_NULLABLE |
+-------------------+--------------------+--------------+
| name | CHARACTER VARYING | NO |
| kind | CHARACTER VARYING | NO |
| accessibleScopes | CHARACTER VARYING | NO |
| optionScope | CHARACTER VARYING | NO |
| status | CHARACTER VARYING | NO |
| num_val | BIGINT | YES |
| string_val | CHARACTER VARYING | YES |
| bool_val | BOOLEAN | YES |
| float_val | DOUBLE | YES |
+-------------------+--------------------+--------------+
9 rows selected (0.185 seconds)
0: jdbc:drill:schema=sys> describe options_val;
+-------------------+--------------------+--------------+
| COLUMN_NAME | DATA_TYPE | IS_NULLABLE |
+-------------------+--------------------+--------------+
| name | CHARACTER VARYING | NO |
| kind | CHARACTER VARYING | NO |
| accessibleScopes | CHARACTER VARYING | NO |
| val | CHARACTER VARYING | YES |
| optionScope | CHARACTER VARYING | NO |
+-------------------+--------------------+--------------+
5 rows selected (0.183 seconds)
0: jdbc:drill:schema=sys> describe profiles_json;
+--------------+--------------------+--------------+
| COLUMN_NAME | DATA_TYPE | IS_NULLABLE |
+--------------+--------------------+--------------+
| queryId | CHARACTER VARYING | NO |
| json | CHARACTER VARYING | YES |
+--------------+--------------------+--------------+
2 rows selected (0.727 seconds)
0: jdbc:drill:schema=sys> describe profiles;
+--------------+--------------------+--------------+
| COLUMN_NAME | DATA_TYPE | IS_NULLABLE |
+--------------+--------------------+--------------+
| queryId | CHARACTER VARYING | NO |
| startTime | TIMESTAMP | YES |
| foreman | CHARACTER VARYING | NO |
| fragments | BIGINT | YES |
| user | CHARACTER VARYING | YES |
| queue | CHARACTER VARYING | YES |
| planTime | BIGINT | YES |
| queueTime | BIGINT | YES |
| executeTime | BIGINT | YES |
| totalTime | BIGINT | YES |
| state | CHARACTER VARYING | YES |
| query | CHARACTER VARYING | YES |
+--------------+--------------------+--------------+
12 rows selected (0.309 seconds)
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> System table columns incorrectly marked as non-nullable
> --------------------------------------------------------
>
> Key: DRILL-6588
> URL: https://issues.apache.org/jira/browse/DRILL-6588
> Project: Apache Drill
> Issue Type: Bug
> Components: Metadata
> Affects Versions: 1.13.0
> Reporter: Aman Sinha
> Assignee: Kunal Khatua
> Priority: Major
> Fix For: 1.14.0
>
>
> System table columns can contain null values but they are incorrectly marked
> as non-nullable as shown in example table below:
> {noformat}
> 0: jdbc:drill:drillbit=10.10.10.191> describe sys.boot;
> +-------------------+--------------------+--------------+
> | COLUMN_NAME | DATA_TYPE | IS_NULLABLE |
> +-------------------+--------------------+--------------+
> | name | CHARACTER VARYING | NO |
> | kind | CHARACTER VARYING | NO |
> | accessibleScopes | CHARACTER VARYING | NO |
> | optionScope | CHARACTER VARYING | NO |
> | status | CHARACTER VARYING | NO |
> | num_val | BIGINT | NO |
> | string_val | CHARACTER VARYING | NO |
> | bool_val | BOOLEAN | NO |
> | float_val | DOUBLE | NO |
> +-------------------+--------------------+--------------+{noformat}
>
> Note that several columns are nulls:
> {noformat}
> +---------------------------------------------------+----------+------------------+-------------+--------+---------+------------+----------+-----------+
> | name | kind |
> accessibleScopes | optionScope | status | num_val | string_val | bool_val |
> float_val |
> +---------------------------------------------------+----------+------------------+-------------+--------+---------+------------+----------+-----------+
> drill.exec.options.exec.udf.enable_dynamic_support | BOOLEAN | BOOT | BOOT |
> BOOT | null | null | true | null |{noformat}
>
> Because of the not-null metadata, the predicates on these tables such as
> `WHERE <column> IS NULL` evaluate to FALSE which is incorrect.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
