[jira] [Updated] (HIVE-10231) Compute partition column stats fails if partition col type is date

2015-04-08 Thread Ashutosh Chauhan (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-10231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan updated HIVE-10231:

Affects Version/s: 1.1.0
   0.14.0

 Compute partition column stats fails if partition col type is date
 --

 Key: HIVE-10231
 URL: https://issues.apache.org/jira/browse/HIVE-10231
 Project: Hive
  Issue Type: Bug
  Components: Statistics
Affects Versions: 0.14.0, 1.0.0, 1.1.0
Reporter: Chaoyu Tang
Assignee: Chaoyu Tang
 Fix For: 1.2.0

 Attachments: HIVE-10231.1.patch, HIVE-10231.1.patch, HIVE-10231.patch


 Currently the command analyze table .. partition .. compute statistics for 
 columns may only work for partition column type of string, numeric types, 
 but not others like date. See following case using date as partition coltype:
 {code}
 create table colstatspartdate (key int, value string) partitioned by (ds 
 date, hr int);
 insert into colstatspartdate partition (ds=date '2015-04-02', hr=2) select 
 key, value from src limit 20;
 analyze table colstatspartdate partition (ds=date '2015-04-02', hr=2) compute 
 statistics for columns;
 {code}
 you will get RuntimeException:
 {code}
 FAILED: RuntimeException Cannot convert to Date from: int
 15/04/06 17:30:01 ERROR ql.Driver: FAILED: RuntimeException Cannot convert to 
 Date from: int
 java.lang.RuntimeException: Cannot convert to Date from: int
   at 
 org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorUtils.getDate(PrimitiveObjectInspectorUtils.java:1048)
   at 
 org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorConverter$DateConverter.convert(PrimitiveObjectInspectorConverter.java:264)
   at 
 org.apache.hadoop.hive.ql.optimizer.ConstantPropagateProcFactory.typeCast(ConstantPropagateProcFactory.java:163)
   at 
 org.apache.hadoop.hive.ql.optimizer.ConstantPropagateProcFactory.propagate(ConstantPropagateProcFactory.java:333)
   at 
 org.apache.hadoop.hive.ql.optimizer.ConstantPropagateProcFactory.foldExpr(ConstantPropagateProcFactory.java:242)
 
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-10231) Compute partition column stats fails if partition col type is date

2015-04-07 Thread Chaoyu Tang (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-10231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chaoyu Tang updated HIVE-10231:
---
Attachment: HIVE-10231.1.patch

These failures seem not relevant to the patch. I am not able to reproduce them 
in my local machine. Reattach to kick off the test build to make sure.

 Compute partition column stats fails if partition col type is date
 --

 Key: HIVE-10231
 URL: https://issues.apache.org/jira/browse/HIVE-10231
 Project: Hive
  Issue Type: Bug
  Components: Statistics
Affects Versions: 1.0.0
Reporter: Chaoyu Tang
Assignee: Chaoyu Tang
 Fix For: 1.2.0

 Attachments: HIVE-10231.1.patch, HIVE-10231.1.patch, HIVE-10231.patch


 Currently the command analyze table .. partition .. compute statistics for 
 columns may only work for partition column type of string, numeric types, 
 but not others like date. See following case using date as partition coltype:
 {code}
 create table colstatspartdate (key int, value string) partitioned by (ds 
 date, hr int);
 insert into colstatspartdate partition (ds=date '2015-04-02', hr=2) select 
 key, value from src limit 20;
 analyze table colstatspartdate partition (ds=date '2015-04-02', hr=2) compute 
 statistics for columns;
 {code}
 you will get RuntimeException:
 {code}
 FAILED: RuntimeException Cannot convert to Date from: int
 15/04/06 17:30:01 ERROR ql.Driver: FAILED: RuntimeException Cannot convert to 
 Date from: int
 java.lang.RuntimeException: Cannot convert to Date from: int
   at 
 org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorUtils.getDate(PrimitiveObjectInspectorUtils.java:1048)
   at 
 org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorConverter$DateConverter.convert(PrimitiveObjectInspectorConverter.java:264)
   at 
 org.apache.hadoop.hive.ql.optimizer.ConstantPropagateProcFactory.typeCast(ConstantPropagateProcFactory.java:163)
   at 
 org.apache.hadoop.hive.ql.optimizer.ConstantPropagateProcFactory.propagate(ConstantPropagateProcFactory.java:333)
   at 
 org.apache.hadoop.hive.ql.optimizer.ConstantPropagateProcFactory.foldExpr(ConstantPropagateProcFactory.java:242)
 
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-10231) Compute partition column stats fails if partition col type is date

2015-04-06 Thread Chaoyu Tang (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-10231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chaoyu Tang updated HIVE-10231:
---
Attachment: HIVE-10231.patch

There is a bug in rewriting the analyze query in ColumnStatsSemanticAnalyzer. 
The date value should be quoted in the rewritten query, but was not. 
In this patch, for the simplicity I quote the partition value in the rewritten 
query regardless of its data type. 

 Compute partition column stats fails if partition col type is date
 --

 Key: HIVE-10231
 URL: https://issues.apache.org/jira/browse/HIVE-10231
 Project: Hive
  Issue Type: Bug
  Components: Statistics
Affects Versions: 1.0.0
Reporter: Chaoyu Tang
Assignee: Chaoyu Tang
 Fix For: 1.2.0

 Attachments: HIVE-10231.patch


 Currently the command analyze table .. partition .. compute statistics for 
 columns may only work for partition column type of string, numeric types, 
 but not others like date. See following case using date as partition coltype:
 {code}
 create table colstatspartdate (key int, value string) partitioned by (ds 
 date, hr int);
 insert into colstatspartdate partition (ds=date '2015-04-02', hr=2) select 
 key, value from src limit 20;
 analyze table colstatspartdate partition (ds=date '2015-04-02', hr=2) compute 
 statistics for columns;
 {code}
 you will get RuntimeException:
 {code}
 FAILED: RuntimeException Cannot convert to Date from: int
 15/04/06 17:30:01 ERROR ql.Driver: FAILED: RuntimeException Cannot convert to 
 Date from: int
 java.lang.RuntimeException: Cannot convert to Date from: int
   at 
 org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorUtils.getDate(PrimitiveObjectInspectorUtils.java:1048)
   at 
 org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorConverter$DateConverter.convert(PrimitiveObjectInspectorConverter.java:264)
   at 
 org.apache.hadoop.hive.ql.optimizer.ConstantPropagateProcFactory.typeCast(ConstantPropagateProcFactory.java:163)
   at 
 org.apache.hadoop.hive.ql.optimizer.ConstantPropagateProcFactory.propagate(ConstantPropagateProcFactory.java:333)
   at 
 org.apache.hadoop.hive.ql.optimizer.ConstantPropagateProcFactory.foldExpr(ConstantPropagateProcFactory.java:242)
 
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-10231) Compute partition column stats fails if partition col type is date

2015-04-06 Thread Chaoyu Tang (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-10231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chaoyu Tang updated HIVE-10231:
---
Attachment: HIVE-10231.1.patch

Thanks [~ashutoshc] for the review. I made the changes and also add more tests 
in the new patch, please take a look, thanks.

 Compute partition column stats fails if partition col type is date
 --

 Key: HIVE-10231
 URL: https://issues.apache.org/jira/browse/HIVE-10231
 Project: Hive
  Issue Type: Bug
  Components: Statistics
Affects Versions: 1.0.0
Reporter: Chaoyu Tang
Assignee: Chaoyu Tang
 Fix For: 1.2.0

 Attachments: HIVE-10231.1.patch, HIVE-10231.patch


 Currently the command analyze table .. partition .. compute statistics for 
 columns may only work for partition column type of string, numeric types, 
 but not others like date. See following case using date as partition coltype:
 {code}
 create table colstatspartdate (key int, value string) partitioned by (ds 
 date, hr int);
 insert into colstatspartdate partition (ds=date '2015-04-02', hr=2) select 
 key, value from src limit 20;
 analyze table colstatspartdate partition (ds=date '2015-04-02', hr=2) compute 
 statistics for columns;
 {code}
 you will get RuntimeException:
 {code}
 FAILED: RuntimeException Cannot convert to Date from: int
 15/04/06 17:30:01 ERROR ql.Driver: FAILED: RuntimeException Cannot convert to 
 Date from: int
 java.lang.RuntimeException: Cannot convert to Date from: int
   at 
 org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorUtils.getDate(PrimitiveObjectInspectorUtils.java:1048)
   at 
 org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorConverter$DateConverter.convert(PrimitiveObjectInspectorConverter.java:264)
   at 
 org.apache.hadoop.hive.ql.optimizer.ConstantPropagateProcFactory.typeCast(ConstantPropagateProcFactory.java:163)
   at 
 org.apache.hadoop.hive.ql.optimizer.ConstantPropagateProcFactory.propagate(ConstantPropagateProcFactory.java:333)
   at 
 org.apache.hadoop.hive.ql.optimizer.ConstantPropagateProcFactory.foldExpr(ConstantPropagateProcFactory.java:242)
 
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)