gehaijiang created HIVE-20253:
---------------------------------
Summary: nativetask cann't working in hive
Key: HIVE-20253
URL: https://issues.apache.org/jira/browse/HIVE-20253
Project: Hive
Issue Type: Wish
Affects Versions: 1.2.1
Environment: hadoop 3.0.3
hive sql:
set
mapreduce.job.map.output.collector.class=org.apache.hadoop.mapred.nativetask.NativeMapOutputCollectorDelegator;
select count(*) from test_cold;
Reporter: gehaijiang
hadoop 3.0.3, Support nativetask.
mapred-site.xml:
<property>
<name>mapreduce.job.map.output.collector.class</name>
<value>org.apache.hadoop.mapred.nativetask.NativeMapOutputCollectorDelegator</value>
</property>
hive sql:
set
mapreduce.job.map.output.collector.class=org.apache.hadoop.mapred.nativetask.NativeMapOutputCollectorDelegator;
select count(*) from test_cold;
URL:
http://0.0.0.0:8088/taskdetails.jsp?jobid=job_1532646043398_0019&tipid=task_1532646043398_0019_m_000000
-----
Diagnostic Messages for this Task:
Error: java.io.IOException: Initialization of all the collectors failed. Error
in last collector was:java.io.IOException: Cannot find serializer for
org.apache.hadoop.hive.ql.io.HiveKey
at org.apache.hadoop.mapred.MapTask.createSortingCollector(MapTask.java:423)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:454)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:349)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:174)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1686)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:168)
Caused by: java.io.IOException: Cannot find serializer for
org.apache.hadoop.hive.ql.io.HiveKey
at
org.apache.hadoop.mapred.nativetask.NativeMapOutputCollectorDelegator.init(NativeMapOutputCollectorDelegator.java:127)
at org.apache.hadoop.mapred.MapTask.createSortingCollector(MapTask.java:408)
... 7 more
2018-07-27 10:08:25,391 ERROR operation.Operation (SQLOperation.java:run(209))
- Error running hive query:
org.apache.hive.service.cli.HiveSQLException: Error while processing statement:
FAILED: Execution Error, return code 2 from
org.apache.hadoop.hive.ql.exec.mr.MapRedTask
at
org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:316)
at
org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:156)
at
org.apache.hive.service.cli.operation.SQLOperation.access$100(SQLOperation.java:71)
at
org.apache.hive.service.cli.operation.SQLOperation$1$1.run(SQLOperation.java:206)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at
org.apache.hive.service.cli.operation.SQLOperation$1.run(SQLOperation.java:218)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
