????????????????????hive-site.xml????hive.aux.jars.path????hadoop-lzo????????jar??????????????????????????build
cube??????????????????????????????
yarn????????????????
Error: java.io.IOException: Unable to initialize any output collector at
org.apache.hadoop.mapred.MapTask.createSortingCollector(MapTask.java:412) at
org.apache.hadoop.mapred.MapTask.access$100(MapTask.java:81) at
org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:695) at
org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:767) at
org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) at
org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163) at
java.security.AccessController.doPrivileged(Native Method) at
javax.security.auth.Subject.doAs(Subject.java:422) at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) Container
killed by the ApplicationMaster. Container killed on request. Exit code is 143
Container exited with a non-zero exit code 143
??????????????????????????????
2017-04-19 10:48:15,875 WARN [main] org.apache.hadoop.mapred.MapTask: Unable to
initialize MapOutputCollector org.apache.hadoop.mapred.MapTask$MapOutputBuffer
java.lang.IllegalArgumentException: Compression codec
com.hadoop.compression.lzo.LzoCodec was not found.
at
org.apache.hadoop.mapred.JobConf.getMapOutputCompressorClass(JobConf.java:798)
at
org.apache.hadoop.mapred.MapTask$MapOutputBuffer.init(MapTask.java:1019)
at
org.apache.hadoop.mapred.MapTask.createSortingCollector(MapTask.java:401)
at org.apache.hadoop.mapred.MapTask.access$100(MapTask.java:81)
at
org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:695)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:767)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: java.lang.ClassNotFoundException: Class
com.hadoop.compression.lzo.LzoCodec not found
at
org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1980)
at
org.apache.hadoop.mapred.JobConf.getMapOutputCompressorClass(JobConf.java:796)
... 11 more
????hadoop????????hadoop/bin/hadoop checknative -a
??????????????
17/04/19 09:55:10 WARN bzip2.Bzip2Factory: Failed to load/initialize
native-bzip2 library system-native, will use pure-Java version
17/04/19 09:55:10 INFO zlib.ZlibFactory: Successfully loaded & initialized
native-zlib library
Native library checking:
hadoop: true /home/hadooper/hadoop/lib/native/libhadoop.so.1.0.0
zlib: true /lib64/libz.so.1
snappy: false
lz4: true revision:99
bzip2: false
openssl: true /usr/lib64/libcrypto.so
lzo??????????????????????????????????????????????????
------------------ ???????? ------------------
??????: "35925138";<[email protected]>;
????????: 2017??4??14??(??????) ????4:25
??????: "dev"<[email protected]>;
????: ??????????: ??????????: Error: java.io.IOException: Unable to initialize
any output collector
????????????????????hadoop??????????????????????2047??????????2048
------------------ ???????? ------------------
??????: "roger shi";<[email protected]>;
????????: 2017??4??13??(??????) ????4:30
??????: "dev"<[email protected]>;
????: ????: ??????????: Error: java.io.IOException: Unable to initialize any
output collector
It seems the hadoop environment configuration issue.
One possible reason is the configuration "mapreduce.task.io.sort.mb" of this
failed job is too large. For the detailed information please refer to
https://issues.apache.org/jira/browse/MAPREDUCE-6194.
Bubble up final exception in failures during creation of
...<https://issues.apache.org/jira/browse/MAPREDUCE-6194>
issues.apache.org
MAPREDUCE-5974 added in ability to instantiate multiple OCs, but if none of
them are able to load it "throws" only a final a generic message: "Unable to
initialize ...
________________________________
??????: 35925138 <[email protected]>
????????: 2017??4??13?? 15:13:45
??????: dev
????: ??????????: Error: java.io.IOException: Unable to initialize any output
collector
????????????????????????????????????
------------------ ???????? ------------------
??????: "roger shi";<[email protected]>;
????????: 2017??4??13??(??????) ????2:51
??????: "dev"<[email protected]>;
????: ????: Error: java.io.IOException: Unable to initialize any output
collector
Could you please attach the complete stack trace of the error?
________________________________
??????: 35925138 <[email protected]>
????????: 2017??4??13?? 13:45:09
??????: dev
????: Error: java.io.IOException: Unable to initialize any output collector
????kylin??????1.6.0 hadoop??????2.6.0??build cube????????Step Name:
Redistribute Flat Hive Table ??????????????????
kylin??????????????????
no counters for job job_1492049582227_0011
????hadoop????userlog??????????????????????
2017-04-13 11:23:33,758 ERROR [IPC Server handler 1 on 43772]
org.apache.hadoop.mapred.TaskAttemptListenerImpl: Task:
attempt_1492049582227_0011_m_000000_0 - exited : java.io.IOException: Unable to
initialize any output collector
at
org.apache.hadoop.mapred.MapTask.createSortingCollector(MapTask.java:412)
at org.apache.hadoop.mapred.MapTask.access$100(MapTask.java:81)
at
org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:695)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:767)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
??????????????????????????????????????????
