[
https://issues.apache.org/jira/browse/HIVE-20098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andy Rosa updated HIVE-20098:
-----------------------------
Description:
The issue reproduces only for a date column for a partitioned table. It
reproduces only if the date column has all the values set to null, and if the
partition is not empty.
Here is a quick reproducer:
{code:java}
CREATE TABLE dummy_table (
c_date DATE,
c_bigint BIGINT
)
PARTITIONED BY (ds STRING);
INSERT OVERWRITE TABLE dummy_table PARTITION (ds='2018-01-01') SELECT CAST(null
AS DATE), CAST(null AS BIGINT) FROM <any non empty table>;
ANALYZE TABLE dummy_table COMPUTE STATISTICS FOR COLUMNS;
DESCRIBE FORMATTED dummy_table.c_bigint PARTITION (ds='2018-01-01');
DESCRIBE FORMATTED dummy_table.c_date PARTITION (ds='2018-01-01');
{code}
The first `DESCRIBE FORMATTED` statement succeeds, when the second fails with
an `NPE`
It happens because the null check is missing when converting Object from the
ObjectStore to the Thrift object. The null check is missing only in the date
statistics conversion for the partitioned table.
Missing:
[https://github.com/apache/hive/blob/master/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/StatObjectConverter.java#L469]
Present:
https://github.com/apache/hive/blob/master/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/StatObjectConverter.java#L298
was:
The issue reproduces only for a date column for a partitioned table. It
reproduces only if the date column has all the values set to null, and if the
partition is not empty.
Here is a quick reproducer:
{code:java}
CREATE TABLE dummy_table (
c_date DATE,
c_bigint BIGINT
)
PARTITIONED BY (ds STRING);
INSERT OVERWRITE TABLE dummy_table PARTITION (ds='2018-01-01') SELECT CAST(null
AS DATE), CAST(null AS BIGINT) FROM <any non empty table>;
ANALYZE TABLE dummy_table COMPUTE STATISTICS FOR COLUMNS;
DESCRIBE FORMATTED dummy_table.c_bigint PARTITION (ds='2018-01-01');
DESCRIBE FORMATTED dummy_table.c_date PARTITION (ds='2018-01-01');
{code}
The first `DESCRIBE FORMATTED` statement succeeds, when the second fails with
an `NPE`
It happens because the null check is missing when converting Object from the
ObjectStore to the Thrift object. The null check is missing only in the date
statistics conversion for the partitioned table.
Missing:
[https://github.com/apache/hive/blob/master/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/StatObjectConverter.java#L469]
Present:
[https://github.com/apache/hive/blob/master/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/StatObjectConverter.java#L558]
> NPE when getting Date column statistics
> ---------------------------------------
>
> Key: HIVE-20098
> URL: https://issues.apache.org/jira/browse/HIVE-20098
> Project: Hive
> Issue Type: Bug
> Components: Metastore, Standalone Metastore
> Affects Versions: 1.2.1, 4.0.0
> Environment: Tested on versions `1.2.1` and the latest 4.0.0-SNAPSHOT
> Reporter: Andy Rosa
> Priority: Major
> Attachments:
> 0001-Fix-NPE-when-getting-statistics-for-date-column.patch
>
>
> The issue reproduces only for a date column for a partitioned table. It
> reproduces only if the date column has all the values set to null, and if the
> partition is not empty.
> Here is a quick reproducer:
>
>
> {code:java}
> CREATE TABLE dummy_table (
> c_date DATE,
> c_bigint BIGINT
> )
> PARTITIONED BY (ds STRING);
> INSERT OVERWRITE TABLE dummy_table PARTITION (ds='2018-01-01') SELECT
> CAST(null AS DATE), CAST(null AS BIGINT) FROM <any non empty table>;
> ANALYZE TABLE dummy_table COMPUTE STATISTICS FOR COLUMNS;
> DESCRIBE FORMATTED dummy_table.c_bigint PARTITION (ds='2018-01-01');
> DESCRIBE FORMATTED dummy_table.c_date PARTITION (ds='2018-01-01');
> {code}
>
>
> The first `DESCRIBE FORMATTED` statement succeeds, when the second fails with
> an `NPE`
>
> It happens because the null check is missing when converting Object from the
> ObjectStore to the Thrift object. The null check is missing only in the date
> statistics conversion for the partitioned table.
> Missing:
> [https://github.com/apache/hive/blob/master/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/StatObjectConverter.java#L469]
> Present:
> https://github.com/apache/hive/blob/master/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/StatObjectConverter.java#L298
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
