Thoughts anyone? Thanks,
Jason On Tue, Mar 18, 2014 at 2:23 PM, Jason Rosenberg <[email protected]> wrote: > Hi, > > I'm wondering if there is expected performance increases with using the > --direct flag for exporting from hive to mysql. If so, how much speedup? > > Also, I've been getting lock contention errors during export, and I'm > wondering if these are less likely using --direct mode? E.g. I'm getting > these sorts of exceptions on the sqoop console: > > 14/03/18 14:44:15 INFO mapred.JobClient: Task Id : > attempt_201403180842_0202_m_000002_1, Status : FAILED > java.io.IOException: Can't export data, please check failed map task logs > at > org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:112) > at > org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:39) > at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:140) > at > org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64) > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:672) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:330) > at org.apache.hadoop.mapred.Child$4.run(Child.java:268) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408) > at org.apache.hadoop.mapred.Child.main(Child.java:262) > Caused by: java.io.IOException: java.sql.BatchUpdateException: Deadlock > found when trying to get lock; try restarting transaction > at > org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.write(AsyncSqlRecordWr > > > Thanks, > > Jason >
