[
https://issues.apache.org/jira/browse/HIVE-5240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13764104#comment-13764104
]
Prasanth J commented on HIVE-5240:
----------------------------------
just noticed that the earlier version of the patch missed out two small issues.
issue 1: case insensitive comparison of specified column name with partition
name
{code}hive> analyze table qlog_1m_part partition(year=5) compute statistics for
columns YEAR,month,week,type;
FAILED: IndexOutOfBoundsException Index: 1, Size: 0{code}
issue 2: if partition name is not explicitly specified in the analyze query,
then table level statistics will be computed. table level statistics on
partition column is also not supported yet. Following query illustrates this
case (it is same as the query in description except that partition is NOT
explicitly specified)
{code}hive> analyze table qlog_1m_part compute statistics for columns
year,month,week,type;
FAILED: IndexOutOfBoundsException Index: 1, Size: 0{code}
This new patch addresses these two issues.
> Column statistics on a partitioned column should fail early with proper error
> message
> -------------------------------------------------------------------------------------
>
> Key: HIVE-5240
> URL: https://issues.apache.org/jira/browse/HIVE-5240
> Project: Hive
> Issue Type: Bug
> Components: Statistics
> Affects Versions: 0.12.0
> Reporter: Prasanth J
> Assignee: Prasanth J
> Labels: statistics
> Fix For: 0.12.0
>
> Attachments: HIVE-5240.2.txt, HIVE-5240.3.txt, HIVE-5240.txt
>
>
> When computing column statistics on a partitioned table, if one of the
> columns equals the partitioned column then IndexOutOfBoundsException is
> thrown.
> Following analyze query throws IndexOutOfBoundsException during semantic
> analysis phase
> {code}hive> analyze table qlog_1m_part partition(year=5) compute statistics
> for columns year,month,week,type;
> FAILED: IndexOutOfBoundsException Index: 1, Size: 0{code}
> If the partitioned column is specified at last like below then the same
> exception is thrown at runtime
> {code}hive> analyze table qlog_1m_part partition(year=5) compute statistics
> for columns month,week,type,year;
> Hadoop job information for null: number of mappers: 0; number of reducers: 0
> 2013-09-06 18:05:06,587 null map = 0%, reduce = 100%
> Ended Job = job_local861862820_0001
> Execution completed successfully
> Mapred Local Task Succeeded . Convert the Join into MapJoin
> java.lang.IndexOutOfBoundsException: Index: 3, Size: 3
> at java.util.LinkedList.entry(LinkedList.java:365)
> at java.util.LinkedList.get(LinkedList.java:315)
> at
> org.apache.hadoop.hive.ql.exec.ColumnStatsTask.constructColumnStatsFromPackedRow(ColumnStatsTask.java:262)
> at
> org.apache.hadoop.hive.ql.exec.ColumnStatsTask.persistPartitionStats(ColumnStatsTask.java:302)
> at
> org.apache.hadoop.hive.ql.exec.ColumnStatsTask.execute(ColumnStatsTask.java:345)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:151)
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:65)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1407)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1187)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1017)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:885)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:259)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:413)
> at
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:781)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:675)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:614)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:160)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
