[
https://issues.apache.org/jira/browse/MAPREDUCE-6671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oleksiy Sayankin updated MAPREDUCE-6671:
----------------------------------------
Description:
Execute in Hive
{code}
hive> SET dfs.block.size=32000000;
hive> select count(*) from test;
{code}
See logs
{code}
Query ID = vagrant_20160408135656_fd1937b3-b330-4d54-842a-0f3ec544ceea
Total jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
set mapreduce.job.reduces=<number>
Starting Job = job_1460123221842_0001, Tracking URL =
http://cdh-master:8088/proxy/application_1460123221842_0001/
Kill Command = /usr/lib/hadoop/bin/hadoop job -kill job_1460123221842_0001
Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 1
2016-04-08 13:57:18,494 Stage-1 map = 0%, reduce = 0%
2016-04-08 13:58:06,821 Stage-1 map = 100%, reduce = 100%
Ended Job = job_1460123221842_0001 with errors
Error during job, obtaining debugging information...
Examining task ID: task_1460123221842_0001_m_000000 (and more) from job
job_1460123221842_0001
Task with the most failures(4):
-----
Task ID:
task_1460123221842_0001_m_000000
URL:
http://cdh-master:8088/taskdetails.jsp?jobid=job_1460123221842_0001&tipid=task_1460123221842_0001_m_000000
-----
Diagnostic Messages for this Task:
Exception from container-launch.
Container id: container_1460123221842_0001_01_000005
Exit code: 1
Stack trace: ExitCodeException exitCode=1:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:561)
at org.apache.hadoop.util.Shell.run(Shell.java:478)
at
org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:738)
at
org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:210)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Container exited with a non-zero exit code 1
FAILED: Execution Error, return code 2 from
org.apache.hadoop.hive.ql.exec.mr.MapRedTask
MapReduce Jobs Launched:
Stage-Stage-1: Map: 1 Reduce: 1 HDFS Read: 0 HDFS Write: 0 FAIL
Total MapReduce CPU Time Spent: 0 msec
{code}
We need to have more informative error message here
was:
Execute in Hive
{code}
hive> SET dfs.block.size=32000000;
hive> select count(*) from test;
{code}
Query ID = vagrant_20160408135656_fd1937b3-b330-4d54-842a-0f3ec544ceea
Total jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
set mapreduce.job.reduces=<number>
Starting Job = job_1460123221842_0001, Tracking URL =
http://cdh-master:8088/proxy/application_1460123221842_0001/
Kill Command = /usr/lib/hadoop/bin/hadoop job -kill job_1460123221842_0001
Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 1
2016-04-08 13:57:18,494 Stage-1 map = 0%, reduce = 0%
2016-04-08 13:58:06,821 Stage-1 map = 100%, reduce = 100%
Ended Job = job_1460123221842_0001 with errors
Error during job, obtaining debugging information...
Examining task ID: task_1460123221842_0001_m_000000 (and more) from job
job_1460123221842_0001
Task with the most failures(4):
-----
Task ID:
task_1460123221842_0001_m_000000
URL:
http://cdh-master:8088/taskdetails.jsp?jobid=job_1460123221842_0001&tipid=task_1460123221842_0001_m_000000
-----
Diagnostic Messages for this Task:
Exception from container-launch.
Container id: container_1460123221842_0001_01_000005
Exit code: 1
Stack trace: ExitCodeException exitCode=1:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:561)
at org.apache.hadoop.util.Shell.run(Shell.java:478)
at
org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:738)
at
org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:210)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Container exited with a non-zero exit code 1
FAILED: Execution Error, return code 2 from
org.apache.hadoop.hive.ql.exec.mr.MapRedTask
MapReduce Jobs Launched:
Stage-Stage-1: Map: 1 Reduce: 1 HDFS Read: 0 HDFS Write: 0 FAIL
Total MapReduce CPU Time Spent: 0 msec
> Incorrect error message while setting dfs.block.size to wrong value
> -------------------------------------------------------------------
>
> Key: MAPREDUCE-6671
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6671
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Reporter: Oleksiy Sayankin
> Assignee: Oleksiy Sayankin
> Attachments: MAPREDUCE-6671.1.patch
>
>
> Execute in Hive
> {code}
> hive> SET dfs.block.size=32000000;
> hive> select count(*) from test;
> {code}
> See logs
> {code}
> Query ID = vagrant_20160408135656_fd1937b3-b330-4d54-842a-0f3ec544ceea
> Total jobs = 1
> Launching Job 1 out of 1
> Number of reduce tasks determined at compile time: 1
> In order to change the average load for a reducer (in bytes):
> set hive.exec.reducers.bytes.per.reducer=<number>
> In order to limit the maximum number of reducers:
> set hive.exec.reducers.max=<number>
> In order to set a constant number of reducers:
> set mapreduce.job.reduces=<number>
> Starting Job = job_1460123221842_0001, Tracking URL =
> http://cdh-master:8088/proxy/application_1460123221842_0001/
> Kill Command = /usr/lib/hadoop/bin/hadoop job -kill job_1460123221842_0001
> Hadoop job information for Stage-1: number of mappers: 1; number of reducers:
> 1
> 2016-04-08 13:57:18,494 Stage-1 map = 0%, reduce = 0%
> 2016-04-08 13:58:06,821 Stage-1 map = 100%, reduce = 100%
> Ended Job = job_1460123221842_0001 with errors
> Error during job, obtaining debugging information...
> Examining task ID: task_1460123221842_0001_m_000000 (and more) from job
> job_1460123221842_0001
> Task with the most failures(4):
> -----
> Task ID:
> task_1460123221842_0001_m_000000
> URL:
>
> http://cdh-master:8088/taskdetails.jsp?jobid=job_1460123221842_0001&tipid=task_1460123221842_0001_m_000000
> -----
> Diagnostic Messages for this Task:
> Exception from container-launch.
> Container id: container_1460123221842_0001_01_000005
> Exit code: 1
> Stack trace: ExitCodeException exitCode=1:
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:561)
> at org.apache.hadoop.util.Shell.run(Shell.java:478)
> at
> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:738)
> at
> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:210)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Container exited with a non-zero exit code 1
> FAILED: Execution Error, return code 2 from
> org.apache.hadoop.hive.ql.exec.mr.MapRedTask
> MapReduce Jobs Launched:
> Stage-Stage-1: Map: 1 Reduce: 1 HDFS Read: 0 HDFS Write: 0 FAIL
> Total MapReduce CPU Time Spent: 0 msec
> {code}
> We need to have more informative error message here
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
