[jira] [Updated] (HIVE-20098) Statistics: NPE when getting Date column partition statistics

2018-07-06 Thread Ashutosh Chauhan (JIRA) 


 [ 
https://issues.apache.org/jira/browse/HIVE-20098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan updated HIVE-20098:

   Resolution: Fixed
Fix Version/s: 4.0.0
   Status: Resolved  (was: Patch Available)

Pushed to master. Thanks, Andy!

> Statistics: NPE when getting Date column partition statistics
> -
>
> Key: HIVE-20098
> URL: https://issues.apache.org/jira/browse/HIVE-20098
> Project: Hive
>  Issue Type: Bug
>  Components: Metastore, Standalone Metastore
>Affects Versions: 1.2.1, 4.0.0
> Environment: Tested on versions `1.2.1` and the latest 4.0.0-SNAPSHOT
>Reporter: Andy Rosa
>Assignee: Andy Rosa
>Priority: Major
>  Labels: Branch3Candidate
> Fix For: 4.0.0
>
> Attachments: 
> 0001-Fix-NPE-when-getting-statistics-for-date-column.patch, HIVE-20098.1.patch
>
>
> The issue reproduces only for a date column for a partitioned table. It 
> reproduces only if the date column has all the values set to null, and if the 
> partition is not empty.
> Here is a quick reproducer:
>  
>  
> {code:java}
> CREATE TABLE dummy_table (
> c_date DATE,
> c_bigint BIGINT
> )
> PARTITIONED BY (ds STRING);
> INSERT OVERWRITE TABLE dummy_table PARTITION (ds='2018-01-01') SELECT 
> CAST(null AS DATE), CAST(null AS BIGINT) FROM ;
> ANALYZE TABLE dummy_table COMPUTE STATISTICS FOR COLUMNS;
> DESCRIBE FORMATTED dummy_table.c_bigint PARTITION (ds='2018-01-01');
> DESCRIBE FORMATTED dummy_table.c_date PARTITION (ds='2018-01-01');
> {code}
>  
>  
> The first `DESCRIBE FORMATTED` statement succeeds, when the second fails with 
> an `NPE`
>  
> It happens because the null check is missing when converting Object from the 
> ObjectStore to the Thrift object. The null check is missing only in the date 
> statistics conversion for the partitioned table. 
> Missing: 
> [https://github.com/apache/hive/blob/master/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/StatObjectConverter.java#L469]
> Present: 
> https://github.com/apache/hive/blob/master/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/StatObjectConverter.java#L298
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-20098) Statistics: NPE when getting Date column partition statistics

2018-07-06 Thread Gopal V (JIRA) 


 [ 
https://issues.apache.org/jira/browse/HIVE-20098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gopal V updated HIVE-20098:
---
Labels: Branch3Candidate  (was: )

> Statistics: NPE when getting Date column partition statistics
> -
>
> Key: HIVE-20098
> URL: https://issues.apache.org/jira/browse/HIVE-20098
> Project: Hive
>  Issue Type: Bug
>  Components: Metastore, Standalone Metastore
>Affects Versions: 1.2.1, 4.0.0
> Environment: Tested on versions `1.2.1` and the latest 4.0.0-SNAPSHOT
>Reporter: Andy Rosa
>Assignee: Andy Rosa
>Priority: Major
>  Labels: Branch3Candidate
> Attachments: 
> 0001-Fix-NPE-when-getting-statistics-for-date-column.patch, HIVE-20098.1.patch
>
>
> The issue reproduces only for a date column for a partitioned table. It 
> reproduces only if the date column has all the values set to null, and if the 
> partition is not empty.
> Here is a quick reproducer:
>  
>  
> {code:java}
> CREATE TABLE dummy_table (
> c_date DATE,
> c_bigint BIGINT
> )
> PARTITIONED BY (ds STRING);
> INSERT OVERWRITE TABLE dummy_table PARTITION (ds='2018-01-01') SELECT 
> CAST(null AS DATE), CAST(null AS BIGINT) FROM ;
> ANALYZE TABLE dummy_table COMPUTE STATISTICS FOR COLUMNS;
> DESCRIBE FORMATTED dummy_table.c_bigint PARTITION (ds='2018-01-01');
> DESCRIBE FORMATTED dummy_table.c_date PARTITION (ds='2018-01-01');
> {code}
>  
>  
> The first `DESCRIBE FORMATTED` statement succeeds, when the second fails with 
> an `NPE`
>  
> It happens because the null check is missing when converting Object from the 
> ObjectStore to the Thrift object. The null check is missing only in the date 
> statistics conversion for the partitioned table. 
> Missing: 
> [https://github.com/apache/hive/blob/master/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/StatObjectConverter.java#L469]
> Present: 
> https://github.com/apache/hive/blob/master/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/StatObjectConverter.java#L298
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-20098) Statistics: NPE when getting Date column partition statistics

2018-07-05 Thread Gopal V (JIRA) 


 [ 
https://issues.apache.org/jira/browse/HIVE-20098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gopal V updated HIVE-20098:
---
Status: Patch Available  (was: Open)

> Statistics: NPE when getting Date column partition statistics
> -
>
> Key: HIVE-20098
> URL: https://issues.apache.org/jira/browse/HIVE-20098
> Project: Hive
>  Issue Type: Bug
>  Components: Metastore, Standalone Metastore
>Affects Versions: 1.2.1, 4.0.0
> Environment: Tested on versions `1.2.1` and the latest 4.0.0-SNAPSHOT
>Reporter: Andy Rosa
>Assignee: Gopal V
>Priority: Major
> Attachments: 
> 0001-Fix-NPE-when-getting-statistics-for-date-column.patch, HIVE-20098.1.patch
>
>
> The issue reproduces only for a date column for a partitioned table. It 
> reproduces only if the date column has all the values set to null, and if the 
> partition is not empty.
> Here is a quick reproducer:
>  
>  
> {code:java}
> CREATE TABLE dummy_table (
> c_date DATE,
> c_bigint BIGINT
> )
> PARTITIONED BY (ds STRING);
> INSERT OVERWRITE TABLE dummy_table PARTITION (ds='2018-01-01') SELECT 
> CAST(null AS DATE), CAST(null AS BIGINT) FROM ;
> ANALYZE TABLE dummy_table COMPUTE STATISTICS FOR COLUMNS;
> DESCRIBE FORMATTED dummy_table.c_bigint PARTITION (ds='2018-01-01');
> DESCRIBE FORMATTED dummy_table.c_date PARTITION (ds='2018-01-01');
> {code}
>  
>  
> The first `DESCRIBE FORMATTED` statement succeeds, when the second fails with 
> an `NPE`
>  
> It happens because the null check is missing when converting Object from the 
> ObjectStore to the Thrift object. The null check is missing only in the date 
> statistics conversion for the partitioned table. 
> Missing: 
> [https://github.com/apache/hive/blob/master/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/StatObjectConverter.java#L469]
> Present: 
> https://github.com/apache/hive/blob/master/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/StatObjectConverter.java#L298
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-20098) Statistics: NPE when getting Date column partition statistics

2018-07-05 Thread Gopal V (JIRA) 


 [ 
https://issues.apache.org/jira/browse/HIVE-20098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gopal V updated HIVE-20098:
---
Attachment: HIVE-20098.1.patch

> Statistics: NPE when getting Date column partition statistics
> -
>
> Key: HIVE-20098
> URL: https://issues.apache.org/jira/browse/HIVE-20098
> Project: Hive
>  Issue Type: Bug
>  Components: Metastore, Standalone Metastore
>Affects Versions: 1.2.1, 4.0.0
> Environment: Tested on versions `1.2.1` and the latest 4.0.0-SNAPSHOT
>Reporter: Andy Rosa
>Assignee: Gopal V
>Priority: Major
> Attachments: 
> 0001-Fix-NPE-when-getting-statistics-for-date-column.patch, HIVE-20098.1.patch
>
>
> The issue reproduces only for a date column for a partitioned table. It 
> reproduces only if the date column has all the values set to null, and if the 
> partition is not empty.
> Here is a quick reproducer:
>  
>  
> {code:java}
> CREATE TABLE dummy_table (
> c_date DATE,
> c_bigint BIGINT
> )
> PARTITIONED BY (ds STRING);
> INSERT OVERWRITE TABLE dummy_table PARTITION (ds='2018-01-01') SELECT 
> CAST(null AS DATE), CAST(null AS BIGINT) FROM ;
> ANALYZE TABLE dummy_table COMPUTE STATISTICS FOR COLUMNS;
> DESCRIBE FORMATTED dummy_table.c_bigint PARTITION (ds='2018-01-01');
> DESCRIBE FORMATTED dummy_table.c_date PARTITION (ds='2018-01-01');
> {code}
>  
>  
> The first `DESCRIBE FORMATTED` statement succeeds, when the second fails with 
> an `NPE`
>  
> It happens because the null check is missing when converting Object from the 
> ObjectStore to the Thrift object. The null check is missing only in the date 
> statistics conversion for the partitioned table. 
> Missing: 
> [https://github.com/apache/hive/blob/master/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/StatObjectConverter.java#L469]
> Present: 
> https://github.com/apache/hive/blob/master/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/StatObjectConverter.java#L298
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-20098) Statistics: NPE when getting Date column partition statistics

2018-07-05 Thread Gopal V (JIRA) 


 [ 
https://issues.apache.org/jira/browse/HIVE-20098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gopal V updated HIVE-20098:
---
Summary: Statistics: NPE when getting Date column partition statistics  
(was: NPE when getting Date column statistics)

> Statistics: NPE when getting Date column partition statistics
> -
>
> Key: HIVE-20098
> URL: https://issues.apache.org/jira/browse/HIVE-20098
> Project: Hive
>  Issue Type: Bug
>  Components: Metastore, Standalone Metastore
>Affects Versions: 1.2.1, 4.0.0
> Environment: Tested on versions `1.2.1` and the latest 4.0.0-SNAPSHOT
>Reporter: Andy Rosa
>Priority: Major
> Attachments: 
> 0001-Fix-NPE-when-getting-statistics-for-date-column.patch
>
>
> The issue reproduces only for a date column for a partitioned table. It 
> reproduces only if the date column has all the values set to null, and if the 
> partition is not empty.
> Here is a quick reproducer:
>  
>  
> {code:java}
> CREATE TABLE dummy_table (
> c_date DATE,
> c_bigint BIGINT
> )
> PARTITIONED BY (ds STRING);
> INSERT OVERWRITE TABLE dummy_table PARTITION (ds='2018-01-01') SELECT 
> CAST(null AS DATE), CAST(null AS BIGINT) FROM ;
> ANALYZE TABLE dummy_table COMPUTE STATISTICS FOR COLUMNS;
> DESCRIBE FORMATTED dummy_table.c_bigint PARTITION (ds='2018-01-01');
> DESCRIBE FORMATTED dummy_table.c_date PARTITION (ds='2018-01-01');
> {code}
>  
>  
> The first `DESCRIBE FORMATTED` statement succeeds, when the second fails with 
> an `NPE`
>  
> It happens because the null check is missing when converting Object from the 
> ObjectStore to the Thrift object. The null check is missing only in the date 
> statistics conversion for the partitioned table. 
> Missing: 
> [https://github.com/apache/hive/blob/master/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/StatObjectConverter.java#L469]
> Present: 
> https://github.com/apache/hive/blob/master/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/StatObjectConverter.java#L298
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)