GitHub user chenshaoxing opened a pull request:
https://github.com/apache/sqoop/pull/5
ClassWriter list not long size error
hive export mysql function:
hive table data ï¼
id name age
1,chenshaoxing,20
2,hugo
second record age filed is null
exec:
sqoop export --connect jdbc:mysql://172.20.4.34:3306/test1 --username root
--password root --table t_part --export-dir /csx/hive/table
--input-fields-terminated-by ',' --input-null-string '\\N'
--input-null-non-string '\\N'
RunTimeException:
Error: java.io.IOException: Can't export data, please check failed map task
logs
at
org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:112)
at
org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:39)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
at
org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1554)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
Caused by: java.lang.RuntimeException: Can't parse input data: 'csx'
at t_part.__loadFromFields(t_part.java:292)
at t_part.parse(t_part.java:230)
at
org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:83)
... 10 more
Caused by: java.util.NoSuchElementException
at java.util.ArrayList$Itr.next(ArrayList.java:834)
at t_part.__loadFromFields(t_part.java:287)
... 12 more
genarate java class:
private void __loadFromFields(List<String> fields) {
Iterator<String> __it = fields.listIterator();
String __cur_str = null;
try {
// if(__it.hasNext()){
__cur_str = __it.next();
if (__cur_str.equals("\\N") || __cur_str.length() == 0) { this.id =
null; } else {
this.id = Long.valueOf(__cur_str);
}
// }
// if(__it.hasNext()){
__cur_str = __it.next();
if (__cur_str.equals("\\N")) { this.name = null; } else {
this.name = __cur_str;
}
// }
// this throw Exception
//if(__it.hasNext()){
__cur_str = __it.next(); //NoSuchElementException
if (__cur_str.equals("\\N")) { this.age = null; } else {
this.age = __cur_str;
}
//}
first line list size = 3, second line list size = 2;
so add if(__it.hasNext()){ }
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/chenshaoxing/sqoop trunk
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/sqoop/pull/5.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #5
----
commit d18b3fe2853fd1ce7783932c46b5cee475b3af9f
Author: chenshaoxing <[email protected]>
Date: 2015-01-29T06:48:33Z
ClassWriter list not long size error
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
