[
https://issues.apache.org/jira/browse/HIVE-5124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13747102#comment-13747102
]
cyril liao commented on HIVE-5124:
----------------------------------
The sql is :
SELECT channeled,
max(VV)AS vv,
max(FUV) AS FUV,
max(PV) AS PV,
max(UV) AS UV
FROM
(
SELECT channeled,
sum(CASE WHEN TYPE = 1 THEN a else cast( 0 as bigint) END) AS VV,
sum(CASE WHEN TYPE = 1 THEN b else cast( 0 as bigint) END) AS FUV,
sum(CASE WHEN TYPE = 2 THEN a else cast (0 as bigint) END) AS PV,
sum(CASE WHEN TYPE = 2 THEN b else cast (0 as bigint) END) AS UV
FROM
(SELECT count(uid) AS a, count(DISTINCT uid) AS b, TYPE, channeled
FROM
(SELECT uid,
channeled,
TYPE
FROM
(SELECT uid,
parse_url(url,'QUERY','channeled') as channeled,
1 AS TYPE
FROM t_html5_vv
WHERE p_day = ${idate}
UNION ALL
SELECT uid,
parse_url(url,'QUERY','channeled') as
channeled,
2 AS TYPE
FROM t_html5_pv
WHERE p_day = ${idate}
)tmp
where channeled is not null and channeled <> ''
) tmp2
GROUP BY channeled,TYPE
)tmp3
GROUP BY channeled)tmp4
GROUP BY channeled
I want to get uv and fuv from different table,t_html5_vv and t_html5_pv ,and
combine the result in one row. The default hive.map.aggr argument in
hive-site.xml is setted to true,and the sql goes prefect.But the exception is
thrown when i set hive.map.aggr= false.
> group by without map aggregation lead to mapreduce exception
> ------------------------------------------------------------
>
> Key: HIVE-5124
> URL: https://issues.apache.org/jira/browse/HIVE-5124
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.11.0
> Reporter: cyril liao
> Assignee: Bing Li
>
> On my environment, the same query but diffent by seting hive.map.aggr with
> true or flase,produce different result.
> With hive.map.aggr=false,tasktracker report the following exception:
> java.lang.RuntimeException: Error in configuring object
> at
> org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
> at
> org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
> at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
> at
> org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:485)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:420)
> at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1083)
> at org.apache.hadoop.mapred.Child.main(Child.java:249)
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
> ... 9 more
> Caused by: java.lang.RuntimeException: Reduce operator initialization failed
> at
> org.apache.hadoop.hive.ql.exec.ExecReducer.configure(ExecReducer.java:160)
> ... 14 more
> Caused by: java.lang.RuntimeException: cannot find field value from [0:_col0,
> 1:_col1, 2:_col2, 3:_col3]
> at
> org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.getStandardStructFieldRef(ObjectInspectorUtils.java:366)
> at
> org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.getStructFieldRef(StandardStructObjectInspector.java:143)
> at
> org.apache.hadoop.hive.ql.exec.ExprNodeColumnEvaluator.initialize(ExprNodeColumnEvaluator.java:82)
> at
> org.apache.hadoop.hive.ql.exec.GroupByOperator.initializeOp(GroupByOperator.java:299)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:451)
> at
> org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:407)
> at
> org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:62)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:451)
> at
> org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:407)
> at
> org.apache.hadoop.hive.ql.exec.GroupByOperator.initializeOp(GroupByOperator.java:438)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375)
> at
> org.apache.hadoop.hive.ql.exec.ExecReducer.configure(ExecReducer.java:153)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
