[
https://issues.apache.org/jira/browse/HADOOP-13774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Omar Bouras updated HADOOP-13774:
---------------------------------
Attachment: Error_message.png
Failed_app_exec.png
In several times, the job of mapreduce ended with success. However the Job
which launched it failed [^Failed_app_exec.png]
{code}
16/11/01 10:33:16 INFO client.RMProxy: Connecting to ResourceManager at
/0.0.0.0:8032
16/11/01 10:33:18 INFO input.FileInputFormat: Total input paths to process : 4
16/11/01 10:33:18 INFO mapreduce.JobSubmitter: number of splits:4
16/11/01 10:33:18 INFO mapreduce.JobSubmitter: Submitting tokens for job:
job_1477992554679_0002
16/11/01 10:33:18 INFO mapreduce.JobSubmitter: Kind: YARN_AM_RM_TOKEN, Service:
, Ident: (appAttemptId { application_id { id: 1 cluster_timestamp:
1477992554679 } attemptId: 1 } keyId: 1178737823)
16/11/01 10:33:19 INFO impl.YarnClientImpl: Submitted application
application_1477992554679_0002
16/11/01 10:33:19 INFO mapreduce.Job: The url to track the job:
http://MEA-029-L:8088/proxy/application_1477992554679_0002/
16/11/01 10:33:19 INFO mapreduce.Job: Running job: job_1477992554679_0002
16/11/01 10:46:00 INFO mapreduce.Job: Job job_1477992554679_0002 running in
uber mode : false
16/11/01 10:46:00 INFO mapreduce.Job: map 0% reduce 0%
16/11/01 10:46:10 INFO mapreduce.Job: map 75% reduce 0%
16/11/01 10:46:11 INFO mapreduce.Job: map 100% reduce 0%
16/11/01 10:46:15 INFO mapreduce.Job: map 100% reduce 100%
16/11/01 10:46:16 INFO mapreduce.Job: Job job_1477992554679_0002 completed
successfully
16/11/01 10:46:16 INFO mapreduce.Job: Counters: 49
File System Counters
FILE: Number of bytes read=47766
FILE: Number of bytes written=688270
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
HDFS: Number of bytes read=105882
HDFS: Number of bytes written=33400
HDFS: Number of read operations=15
HDFS: Number of large read operations=0
HDFS: Number of write operations=2
Job Counters
Launched map tasks=4
Launched reduce tasks=1
Data-local map tasks=4
Total time spent by all maps in occupied slots (ms)=30786
Total time spent by all reduces in occupied slots (ms)=2869
Total time spent by all map tasks (ms)=30786
Total time spent by all reduce tasks (ms)=2869
Total vcore-milliseconds taken by all map tasks=30786
Total vcore-milliseconds taken by all reduce tasks=2869
Total megabyte-milliseconds taken by all map tasks=31524864
Total megabyte-milliseconds taken by all reduce tasks=2937856
Map-Reduce Framework
Map input records=2161
Map output records=14799
Map output bytes=161969
Map output materialized bytes=47784
Input split bytes=413
Combine input records=14799
Combine output records=2988
Reduce input groups=2667
Reduce shuffle bytes=47784
Reduce input records=2988
Reduce output records=2667
Spilled Records=5976
Shuffled Maps =4
Failed Shuffles=0
Merged Map outputs=4
GC time elapsed (ms)=1199
CPU time spent (ms)=5150
Physical memory (bytes) snapshot=1253482496
Virtual memory (bytes) snapshot=9562353664
Total committed heap usage (bytes)=925892608
Shuffle Errors
BAD_ID=0
CONNECTION=0
IO_ERROR=0
WRONG_LENGTH=0
WRONG_MAP=0
WRONG_REDUCE=0
File Input Format Counters
Bytes Read=105469
File Output Format Counters
Bytes Written=33400
{code}
It always end with the same error message [^Error_message.png].
even with increasing the *mapreduce.task.timeout*
{code}
<property>
<name>mapreduce.task.timeout</name>
<value>1800000</value>
<description>The number of milliseconds before a task will be
terminated if it neither reads an input, writes an output, nor
updates its status string. A value of 0 disables the timeout.
</description>
</property>
{code}
> Rest Loaded App fails
> ---------------------
>
> Key: HADOOP-13774
> URL: https://issues.apache.org/jira/browse/HADOOP-13774
> Project: Hadoop Common
> Issue Type: Bug
> Environment: Hadoop Map Reduce REST
> Reporter: Omar Bouras
> Priority: Minor
> Fix For: 2.7.3
>
> Attachments: Error_message.png, Failed_app_exec.png, WordCount.java,
> app.json
>
>
> Hello,
> I am launching an app within a MR REST. This app executes well within Hadoop
> normal invocation. However, when I use the rest
> {code}
> curl -i -X POST -H 'Accept: application/json' -H 'Content-Type:
> application/json' http://localhost:8088/ws/v1/cluster/apps?user.name=exo -d
> @app.json
> {code}
> 6/10/31 21:42:13 INFO client.RMProxy: Connecting to ResourceManager at
> /0.0.0.0:8032
> 16/10/31 21:42:14 INFO input.FileInputFormat: Total input paths to process : 4
> 16/10/31 21:42:14 INFO mapreduce.JobSubmitter: number of splits:4
> 16/10/31 21:42:14 INFO mapreduce.JobSubmitter: Submitting tokens for job:
> job_1477946173138_0003
> 16/10/31 21:42:14 INFO mapreduce.JobSubmitter: Kind: YARN_AM_RM_TOKEN,
> Service: , Ident: (appAttemptId { application_id { id: 3 cluster_timestamp:
> 1477946173138 } attemptId: 1 } keyId: -678745738)
> 16/10/31 21:42:15 INFO impl.YarnClientImpl: Submitted application
> application_1477946173138_0003
> 16/10/31 21:42:15 INFO mapreduce.Job: The url to track the job:
> http://MEA-029-L:8088/proxy/application_1477946173138_0003/
> 16/10/31 21:42:15 INFO mapreduce.Job: Running job: job_1477946173138_0003
> 16/10/31 21:52:53 INFO mapreduce.Job: Job job_1477946173138_0003 running in
> uber mode : false
> 16/10/31 21:52:53 INFO mapreduce.Job: map 0% reduce 0%
> 16/10/31 21:52:53 INFO mapreduce.Job: Job job_1477946173138_0003 failed with
> state FAILED due to: Application application_1477946173138_0003 failed 1
> times due to ApplicationMaster for attempt
> appattempt_1477946173138_0003_000001 timed out. Failing the application.
> 16/10/31 21:52:53 INFO mapreduce.Job: Counters: 0
> {code}
> The main job always ends fail with timeout.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
