Hi
My map job must split the record, so I just set the attribute
“splitor” as the bytes “\001\002” , just like these statements:
StringBuffer ret=new StringBuffer();
ret.append((char)Integer.parseInt("001")).toString();
while it seems that the map/reduce couldn’t resolve and store these
characters. When starting the job ,it goes with this issue
11/07/15 10:56:30 WARN mapred.JobClient: Use GenericOptionsParser for parsing
the arguments. Applications should implement Tool for the same.
11/07/15 10:56:30 INFO lzo.GPLNativeCodeLoader: Loaded native gpl library
11/07/15 10:56:30 INFO lzo.LzoCodec: Successfully loaded & initialized
native-lzo library [hadoop-lzo rev 927df213a90620c11333f9e254cb6816d5dc2b99]
11/07/15 10:56:30 INFO mapred.FileInputFormat: Total input paths to process : 2
11/07/15 10:56:30 INFO mapred.JobClient: Cleaning up the staging area
hdfs://qabst146253:9857/tmp/hadoop-
hbase/mapred/staging/hbase/.staging/job_201107120947_0013
Exception in thread "main" org.apache.hadoop.ipc.RemoteException:
java.io.IOException: java.lang.RuntimeException: org.xml.sax.SAXParseException:
Character
reference "" is an invalid XML character.
at org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3759)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:557)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1415)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1411)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1115)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1409)
Caused by: java.lang.RuntimeException: org.xml.sax.SAXParseException: Character
reference "" is an invalid XML character.
at
org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1393)
at
org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1261)
at
org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1192)
at org.apache.hadoop.conf.Configuration.get(Configuration.java:415)
at
org.apache.hadoop.mapred.JobConf.checkAndWarnDeprecation(JobConf.java:1957)
at org.apache.hadoop.mapred.JobConf.<init>(JobConf.java:386)
at org.apache.hadoop.mapred.JobInProgress.<init>(JobInProgress.java:413)
at org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3757)
... 11 more
Caused by: org.xml.sax.SAXParseException: Character reference "" is an
invalid XML character.
at
com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:249)
at
com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:284)
at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:124)
at
org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1313)
... 18 more
at org.apache.hadoop.ipc.Client.call(Client.java:1104)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:226)
at org.apache.hadoop.mapred.$Proxy1.submitJob(Unknown Source)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:904)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:833)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1115)
at
org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:833)
at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:807)
at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1242)
at com.alibaba.idg.dumpdiff.dumpValidate.main(dumpValidate.java:454)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:186)
________________________________
This email (including any attachments) is confidential and may be legally
privileged. If you received this email in error, please delete it immediately
and do not copy it or use it for any purpose or disclose its contents to any
other person. Thank you.
本电邮(包括任何附件)可能含有机密资料并受法律保护。如您不是正确的收件人,请您立即删除本邮件。请不要将本电邮进行复制并用作任何其他用途、或透露本邮件之内容。谢谢。