Hi All,
I'm working on Hadoop CDH3 U0 and Sqoop CDH3 U2.
I'm trying to export csv files from HDFS to Teradata, it works well with
setting mapper to "1" ( with batch loading of 1000 records at a time ).
when I tried increasing the number of mappers to more than one I'm getting
the following error. Also, is it possible to configure batch size at the
time of export?
sqoop export --verbose --driver com.teradata.jdbc.TeraDriver
--connect jdbc:teradata://xxxx/database=xxxx --username xxxxx --password
xxxxx --table xxxx --export-dir /user/surasani/10minutes.txt
--fields-terminated-by '|' -m 4 --batch
12/01/24 16:17:21 INFO mapred.JobClient: map 3% reduce 0%
12/01/24 16:17:48 INFO mapred.JobClient: Task Id :
attempt_201112211106_68553_m_000001_2, Status : FAILED
*java.io.IOException: java.sql.BatchUpdateException: [Teradata Database]
[TeraJDBC 13.00.00.07] [Error 2631] [SQLState 40001] Transaction ABORTed
due to DeadLock*.
at
com.cloudera.sqoop.mapreduce.AsyncSqlRecordWriter.write(AsyncSqlRecordWriter.java:223)
at
com.cloudera.sqoop.mapreduce.AsyncSqlRecordWriter.write(AsyncSqlRecordWriter.java:49)
at
org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.write(MapTask.java:530)
at
org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
at
com.cloudera.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:82)
at
com.cloudera.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:40)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at
com.cloudera.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:189)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:646)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:322)
at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1115)
at org.apache.hadoop.mapred.Child.main(Child.java:262)
Caused by: java.sql.BatchUpdateException: [Teradata Database] [TeraJDBC
13.00.00.07] [Error 2631] [SQLState 40001] Transaction ABORTed due to
DeadLock.
at
com.teradata.jdbc.jdbc_4.util.ErrorFactory.convertToBatchUpdateException(ErrorFactory.java:116)
at
com.teradata.jdbc.jdbc_4.PreparedStatement.executeBatchDMLArray(PreparedStatement.java:149)
at
com.teradata.jdbc.jdbc_3.ifjdbc_4.TeraLocalPreparedStatement.executeBatch(TeraLocalPreparedStatement.java:299)
at
com.cloudera.sqoop.mapreduce.AsyncSqlOutputFormat$AsyncSqlExecThread.run(AsyncSqlOutputFormat.java:232)
Caused by: com.teradata.jdbc.jdbc_4.util.JDBCException: [Teradata Database]
[TeraJDBC 13.00.00.07] [Error 2631] [SQLState 40001] Transaction ABORTed
due to DeadLock.
at
com.teradata.jdbc.jdbc_4.util.ErrorFactory.makeDatabaseSQLException(ErrorFactory.java:288)
at
com.teradata.jdbc.jdbc_4.statemachine.ReceiveInitSubState.action(ReceiveInitSubState.java:102)
at
com.teradata.jdbc.jdbc_4.statemachine.StatementReceiveState.subStateMachine(StatementReceiveState.java:285)
at
com.teradata.jdbc.jdbc_4.statemachine.StatementReceiveState.action(StatementReceiveState.java:176)
at
com.teradata.jdbc.jdbc_4.statemachine.StatementController.runBody(StatementController.java:108)
at
com.teradata.jdbc.jdbc_4.statemachine.PreparedBatchStatementController.run(PreparedBatchStatementController.java:59)
at
com.teradata.jdbc.jdbc_4.Statement.executeStatement(Statement.java:331)
at
com.teradata.jdbc.jdbc_4.PreparedStatement.executeBatchDMLArray(PreparedStatement.java:138)
... 2 more
12/01/24 16:17:49 INFO mapred.JobClient: map 4% reduce 0%
12/01/24 16:17:52 INFO mapred.JobClient: map 5% reduce 0%
12/01/24 16:21:16 INFO mapred.JobClient: Job complete:
job_201112211106_68553
12/01/24 16:21:16 INFO mapred.JobClient: Counters: 8
12/01/24 16:21:16 INFO mapred.JobClient: Job Counters
12/01/24 16:21:16 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=1709245
12/01/24 16:21:16 INFO mapred.JobClient: Total time spent by all
reduces waiting after reserving slots (ms)=0
12/01/24 16:21:16 INFO mapred.JobClient: Total time spent by all maps
waiting after reserving slots (ms)=0
12/01/24 16:21:16 INFO mapred.JobClient: Rack-local map tasks=1
12/01/24 16:21:16 INFO mapred.JobClient: Launched map tasks=6
12/01/24 16:21:16 INFO mapred.JobClient: Data-local map tasks=3
12/01/24 16:21:16 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=0
12/01/24 16:21:16 INFO mapred.JobClient: Failed map tasks=1
12/01/24 16:21:16 INFO mapreduce.ExportJobBase: Transferred 0 bytes in
862.1506 seconds (0 bytes/sec)
12/01/24 16:21:16 INFO mapreduce.ExportJobBase: Exported 0 records.
12/01/24 16:21:16 ERROR tool.ExportTool: Error during export: Export job
failed!
Thanks,
Srinivas
