-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30345/
-----------------------------------------------------------
Review request for hive.
Bugs: HIVE-9228
https://issues.apache.org/jira/browse/HIVE-9228
Repository: hive-git
Description
-------
The following query with window functions failed. The internal query works fine.
select col1, col2, col3 from (select col1,col2, col3, count(case when col4=1
then 1 end ) over (partition by col1, col2) as col5, row_number() over
(partition by col1, col2 order by col4) as col6 from tab1) t;
HIVE generates an execution plan with 2 jobs.
1. The first job is to basically calculate window function for col5.
2. The second job is to calculate window function for col6 and output.
The plan says the first job outputs the columns (col1, col2, col3, col4) to a
tmp file since only these columns are used in later stage. While, the PTF
operator for the first job outputs (_wcol0, col1, col2, col3, col4) with _wcol0
as the result of the window function even it's not used.
In the second job, the map operator still reads the 4 columns (col1, col2,
col3, col4) from the temp file using the plan. That causes the exception.
Diffs
-----
ql/src/java/org/apache/hadoop/hive/ql/optimizer/ColumnPrunerProcFactory.java
57ce849
ql/src/java/org/apache/hadoop/hive/ql/parse/RowResolver.java 469dc9f
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java c05b2aa
ql/src/test/queries/clientpositive/windowing_windowspec.q 6d8ce67
ql/src/test/results/clientpositive/spark/vectorized_ptf.q.out 020fdff
ql/src/test/results/clientpositive/subquery_notin.q.out 599a61e
ql/src/test/results/clientpositive/subquery_unqualcolumnrefs.q.out 06d5708
ql/src/test/results/clientpositive/tez/vectorized_ptf.q.out a814849
ql/src/test/results/clientpositive/vectorized_ptf.q.out 1e3c43c
ql/src/test/results/clientpositive/windowing_windowspec.q.out 00af6b8
Diff: https://reviews.apache.org/r/30345/diff/
Testing
-------
Thanks,
Navis Ryu
