This is the log.
@shaofengshi
Log Type: stderr
Log Upload Time: Tue Dec 20 14:56:16 +0800 2016
Log Length: 241
log4j:WARN No appenders could be found for logger
(org.apache.hadoop.mapreduce.task.reduce.Fetcher).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for
more info.
Log Type: stdout
Log Upload Time: Tue Dec 20 14:56:16 +0800 2016
Log Length: 0
Log Type: syslog
Log Upload Time: Tue Dec 20 14:56:16 +0800 2016
Log Length: 4477
Showing 4096 bytes of 4477 total. Click here for the full log.
ed
2016-12-20 14:55:29,164 INFO [main] org.apache.hadoop.mapred.YarnChild:
Executing with tokens:
2016-12-20 14:55:29,225 INFO [main] org.apache.hadoop.mapred.YarnChild:
Kind: mapreduce.job, Service: job_1481534908904_0318, Ident:
(org.apache.hadoop.mapreduce.security.token.JobTokenIdentifier@5c3d7f99)
2016-12-20 14:55:29,282 INFO [main] org.apache.hadoop.mapred.YarnChild:
Sleeping for 0ms before retrying again. Got null now.
2016-12-20 14:55:29,760 INFO [main] org.apache.hadoop.mapred.YarnChild:
mapreduce.cluster.local.dir for child:
/udata/yarn/nm/usercache/develop/appcache/application_1481534908904_0318
2016-12-20 14:55:30,415 INFO [main]
org.apache.hadoop.conf.Configuration.deprecation: session.id is deprecated.
Instead, use dfs.metrics.session-id
2016-12-20 14:55:31,005 INFO [main]
org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output
Committer Algorithm version is 1
2016-12-20 14:55:31,049 INFO [main] org.apache.hadoop.mapred.Task: Using
ResourceCalculatorProcessTree : [ ]
2016-12-20 14:55:31,147 INFO [main] org.apache.hadoop.mapred.ReduceTask:
Using ShuffleConsumerPlugin:
org.apache.hadoop.mapreduce.task.reduce.Shuffle@544daf24
2016-12-20 14:55:31,532 INFO [fetcher#1]
org.apache.hadoop.io.compress.CodecPool: Got brand-new decompressor
[.snappy]
2016-12-20 14:55:31,751 INFO [fetcher#9]
org.apache.hadoop.io.compress.CodecPool: Got brand-new decompressor
[.snappy]
2016-12-20 14:55:31,952 INFO [fetcher#8]
org.apache.hadoop.io.compress.CodecPool: Got brand-new decompressor
[.snappy]
2016-12-20 14:55:32,253 INFO [fetcher#10]
org.apache.hadoop.io.compress.CodecPool: Got brand-new decompressor
[.snappy]
2016-12-20 14:55:34,970 WARN [main]
org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException
as:develop (auth:SIMPLE)
cause:org.apache.hadoop.mapreduce.task.reduce.Shuffle$ShuffleError: error in
shuffle in fetcher#10
2016-12-20 14:55:34,975 WARN [main] org.apache.hadoop.mapred.YarnChild:
Exception running child :
org.apache.hadoop.mapreduce.task.reduce.Shuffle$ShuffleError: error in
shuffle in fetcher#10
at org.apache.hadoop.mapreduce.task.reduce.Shuffle.run(Shuffle.java:134)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:376)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: java.lang.OutOfMemoryError: Java heap space
at
org.apache.hadoop.io.BoundedByteArrayOutputStream.<init>(BoundedByteArrayOutputStream.java:56)
at
org.apache.hadoop.io.BoundedByteArrayOutputStream.<init>(BoundedByteArrayOutputStream.java:46)
at
org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput.<init>(InMemoryMapOutput.java:63)
at
org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl.unconditionalReserve(MergeManagerImpl.java:305)
at
org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl.reserve(MergeManagerImpl.java:295)
at
org.apache.hadoop.mapreduce.task.reduce.Fetcher.copyMapOutput(Fetcher.java:514)
at
org.apache.hadoop.mapreduce.task.reduce.Fetcher.copyFromHost(Fetcher.java:336)
at org.apache.hadoop.mapreduce.task.reduce.Fetcher.run(Fetcher.java:193)
2016-12-20 14:55:34,982 INFO [main] org.apache.hadoop.mapred.Task: Runnning
cleanup for the task
2016-12-20 14:55:35,004 WARN [main]
org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: Could not delete
hdfs://dser2:8020/kylin/kylin_metadata/kylin-493a6094-3d42-4ef5-9a7b-debc40731174/ssb_cube/cuboid/_temporary/1/_temporary/attempt_1481534908904_0318_r_000000_0
2016-12-20 14:55:35,109 INFO [main]
org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping ReduceTask
metrics system...
2016-12-20 14:55:35,109 INFO [main]
org.apache.hadoop.metrics2.impl.MetricsSystemImpl: ReduceTask metrics system
stopped.
2016-12-20 14:55:35,109 INFO [main]
org.apache.hadoop.metrics2.impl.MetricsSystemImpl: ReduceTask metrics system
shutdown complete.
Log Type: syslog.shuffle
Log Upload Time: Tue Dec 20 14:56:16 +0800 2016
Log Length: 51522
Showing 4096 bytes of 51522 total. Click here for the full log.
#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1
of 1 to dnode4:13562 to fetcher#2
2016-12-20 14:55:35,099 INFO [fetcher#2]
org.apache.hadoop.mapreduce.task.reduce.Fetcher: for
url=13562/mapOutput?job=job_1481534908904_0318&reduce=0&map=attempt_1481534908904_0318_m_000003_0
sent hash and received reply
2016-12-20 14:55:35,100 INFO [fetcher#2]
org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#2 - MergeManager
returned status WAIT ...
2016-12-20 14:55:35,100 INFO [fetcher#2]
org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: dnode4:13562
freed by fetcher#2 in 2ms
2016-12-20 14:55:35,100 INFO [fetcher#2]
org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning
dnode4:13562 with 1 to fetcher#2
2016-12-20 14:55:35,100 INFO [fetcher#2]
org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of
1 to dnode4:13562 to fetcher#2
2016-12-20 14:55:35,102 INFO [fetcher#2]
org.apache.hadoop.mapreduce.task.reduce.Fetcher: for
url=13562/mapOutput?job=job_1481534908904_0318&reduce=0&map=attempt_1481534908904_0318_m_000003_0
sent hash and received reply
2016-12-20 14:55:35,102 INFO [fetcher#2]
org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#2 - MergeManager
returned status WAIT ...
2016-12-20 14:55:35,102 INFO [fetcher#2]
org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: dnode4:13562
freed by fetcher#2 in 2ms
2016-12-20 14:55:35,102 INFO [fetcher#2]
org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning
dnode4:13562 with 1 to fetcher#2
2016-12-20 14:55:35,102 INFO [fetcher#2]
org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of
1 to dnode4:13562 to fetcher#2
2016-12-20 14:55:35,104 INFO [fetcher#2]
org.apache.hadoop.mapreduce.task.reduce.Fetcher: for
url=13562/mapOutput?job=job_1481534908904_0318&reduce=0&map=attempt_1481534908904_0318_m_000003_0
sent hash and received reply
2016-12-20 14:55:35,104 INFO [fetcher#2]
org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#2 - MergeManager
returned status WAIT ...
2016-12-20 14:55:35,104 INFO [fetcher#2]
org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: dnode4:13562
freed by fetcher#2 in 2ms
2016-12-20 14:55:35,104 INFO [fetcher#2]
org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning
dnode4:13562 with 1 to fetcher#2
2016-12-20 14:55:35,104 INFO [fetcher#2]
org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of
1 to dnode4:13562 to fetcher#2
2016-12-20 14:55:35,106 INFO [fetcher#2]
org.apache.hadoop.mapreduce.task.reduce.Fetcher: for
url=13562/mapOutput?job=job_1481534908904_0318&reduce=0&map=attempt_1481534908904_0318_m_000003_0
sent hash and received reply
2016-12-20 14:55:35,106 INFO [fetcher#2]
org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#2 - MergeManager
returned status WAIT ...
2016-12-20 14:55:35,106 INFO [fetcher#2]
org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: dnode4:13562
freed by fetcher#2 in 2ms
2016-12-20 14:55:35,106 INFO [fetcher#2]
org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning
dnode4:13562 with 1 to fetcher#2
2016-12-20 14:55:35,106 INFO [fetcher#2]
org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of
1 to dnode4:13562 to fetcher#2
2016-12-20 14:55:35,108 INFO [fetcher#2]
org.apache.hadoop.mapreduce.task.reduce.Fetcher: for
url=13562/mapOutput?job=job_1481534908904_0318&reduce=0&map=attempt_1481534908904_0318_m_000003_0
sent hash and received reply
2016-12-20 14:55:35,108 INFO [fetcher#2]
org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#2 - MergeManager
returned status WAIT ...
2016-12-20 14:55:35,108 INFO [fetcher#2]
org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: dnode4:13562
freed by fetcher#2 in 2ms
2016-12-20 14:55:35,109 INFO [fetcher#2]
org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning
dnode4:13562 with 1 to fetcher#2
2016-12-20 14:55:35,109 INFO [fetcher#2]
org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 of
1 to dnode4:13562 to fetcher#2
--
View this message in context:
http://apache-kylin.74782.x6.nabble.com/Failed-with-Build-Cube-in-18-step-tp6700p6729.html
Sent from the Apache Kylin mailing list archive at Nabble.com.
