[ 
https://issues.apache.org/jira/browse/SQOOP-2411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16124140#comment-16124140
 ] 

Sandish Kumar HN commented on SQOOP-2411:
-----------------------------------------

[~anna.szonyi] This is not SQOOP error. it's from MySQL.

need to increase  'net-read-timeout' values in MySQL. otherwise, it breaks at 
[https://github.com/apache/sqoop/blob/trunk/src/java/org/apache/sqoop/mapreduce/MySQLDumpMapper.java#L437-L485
] throws this error "throw new IOException("mysqldump terminated with status "+ 
Integer.toString(result));"

https://github.com/apache/sqoop/blob/trunk/src/java/org/apache/sqoop/mapreduce/MySQLDumpMapper.java#L437-L485

Should we close this?

> Sqoop using '--direct' option fails with mysqldump exit code 2 and 3
> --------------------------------------------------------------------
>
>                 Key: SQOOP-2411
>                 URL: https://issues.apache.org/jira/browse/SQOOP-2411
>             Project: Sqoop
>          Issue Type: Bug
>          Components: connectors/mysql
>    Affects Versions: 1.4.6
>         Environment: Amazon EMR
>            Reporter: Karthick H
>            Assignee: Sandish Kumar HN
>            Priority: Critical
>
> I am running Sqoop in AWS EMR. I am trying to copy a table ~10 GB from MySQL 
> into HDFS.
> I get the following exception
> 15/07/06 12:19:07 INFO mapreduce.Job: Task Id : 
> attempt_1435664372091_0048_m_000000_2, Status : FAILED
> Error: java.io.IOException: mysqldump terminated with status 3
> at org.apache.sqoop.mapreduce.MySQLDumpMapper.map(MySQLDumpMapper.java:485)
> at org.apache.sqoop.mapreduce.MySQLDumpMapper.map(MySQLDumpMapper.java:49)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:152)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:773)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:175)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:170)
> 15/07/06 12:19:07 INFO mapreduce.Job: Task Id : 
> attempt_1435664372091_0048_m_000005_2, Status : FAILED
> Error: java.io.IOException: mysqldump terminated with status 2
> at org.apache.sqoop.mapreduce.MySQLDumpMapper.map(MySQLDumpMapper.java:485)
> at org.apache.sqoop.mapreduce.MySQLDumpMapper.map(MySQLDumpMapper.java:49)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:152)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:773)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:175)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:170)
> 15/07/06 12:19:08 INFO mapreduce.Job:  map 0% reduce 0%
> 15/07/06 12:19:20 INFO mapreduce.Job:  map 25% reduce 0%
> 15/07/06 12:19:22 INFO mapreduce.Job:  map 38% reduce 0%
> 15/07/06 12:19:23 INFO mapreduce.Job:  map 50% reduce 0%
> 15/07/06 12:19:24 INFO mapreduce.Job:  map 75% reduce 0%
> 15/07/06 12:19:25 INFO mapreduce.Job:  map 100% reduce 0%
> 15/07/06 12:23:11 INFO mapreduce.Job: Job job_1435664372091_0048 failed with 
> state FAILED due to: Task failed task_1435664372091_0048_m_000000
> Job failed as tasks failed. failedMaps:1 failedReduces:0
> 15/07/06 12:23:11 INFO mapreduce.Job: Counters: 8
>         Job Counters 
>         Failed map tasks=28
>         Launched map tasks=28
>         Other local map tasks=28
>         Total time spent by all maps in occupied slots (ms)=34760760
>         Total time spent by all reduces in occupied slots (ms)=0
>         Total time spent by all map tasks (ms)=5793460
>         Total vcore-seconds taken by all map tasks=5793460
>         Total megabyte-seconds taken by all map tasks=8342582400
> 15/07/06 12:23:11 WARN mapreduce.Counters: Group FileSystemCounters is 
> deprecated. Use org.apache.hadoop.mapreduce.FileSystemCounter instead
> 15/07/06 12:23:11 INFO mapreduce.ImportJobBase: Transferred 0 bytes in 
> 829.8697 seconds (0 bytes/sec)
> 15/07/06 12:23:11 WARN mapreduce.Counters: Group   
> org.apache.hadoop.mapred.Task$Counter is deprecated. Use 
> org.apache.hadoop.mapreduce.TaskCounter instead
> 15/07/06 12:23:11 INFO mapreduce.ImportJobBase: Retrieved 0 records.
> 15/07/06 12:23:11 ERROR tool.ImportTool: Error during import: Import job 
> failed!
> If I run with out '--direct' option, I get the communication exception as in 
> https://issues.cloudera.org/browse/SQOOP-186
> I have set 'net-write-timeout' and 'net-read-timeout' values in MySQL to 6000.
> My Sqoop command looks like this
> sqoop import -D mapred.task.timeout=0 --fields-terminated-by '\t' 
> --escaped-by '\\' --optionally-enclosed-by '\"' --bindir ./ --connect 
> jdbc:mysql://<remote ip>/<mysql db> --username tuser --password tuser --table 
> table1 --target-dir=/base/table1 --split-by id -m 8 --direct



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Reply via email to