Hi Abe, Thanks for the prompt reply and support. We are using the following sqoop command for code from 1.4.5 branch:
sqoop import -Dmapred.job.queue.name=default --connect jdbc:mysql://localhost/test --username root --password '' --table Temp_Employee --incremental lastmodified --check-column last_updated --last-value "2014-001-01 00:00:00" --hive-import --hive-table="Temp_Employee" -m 1 And we are getting the following exception : 14/09/25 03:35:11 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `Temp_Employee` AS t LIMIT 1 14/09/25 03:35:11 ERROR manager.SqlManager: Error reading from database: java.sql.SQLException: Streaming result set com.mysql.jdbc.RowDataDynamic@69222fc3 is still active. No statements may be issued when any streaming result sets are open and in use on a given connection. Ensure that you have called .close() on any active streaming result sets before attempting more queries. java.sql.SQLException: Streaming result set com.mysql.jdbc.RowDataDynamic@69222fc3 is still active. No statements may be issued when any streaming result sets are open and in use on a given connection. Ensure that you have called .close() on any active streaming result sets before attempting more queries. at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:934) at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:931) at com.mysql.jdbc.MysqlIO.checkForOutstandingStreamingData(MysqlIO.java:2735) at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1899) at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2151) at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2619) at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2569) at com.mysql.jdbc.StatementImpl.executeQuery(StatementImpl.java:1524) at com.mysql.jdbc.ConnectionImpl.getMaxBytesPerChar(ConnectionImpl.java:3003) at com.mysql.jdbc.Field.getMaxBytesPerCharacter(Field.java:602) at com.mysql.jdbc.ResultSetMetaData.getPrecision(ResultSetMetaData.java:445) at org.apache.sqoop.manager.SqlManager.getColumnInfoForRawQuery(SqlManager.java:285) at org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:240) at org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:226) at org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:295) at org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1773) at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1578) at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:96) at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:478) at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:601) at org.apache.sqoop.Sqoop.run(Sqoop.java:143) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227) at org.apache.sqoop.Sqoop.main(Sqoop.java:236) 14/09/25 03:35:11 ERROR tool.ImportTool: Encountered IOException running import job: java.io.IOException: No columns to generate for ClassWriter at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1584) at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:96) at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:478) at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:601) at org.apache.sqoop.Sqoop.run(Sqoop.java:143) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227) at org.apache.sqoop.Sqoop.main(Sqoop.java:236) The same command works on 1.4.6(trunk) and even on 1.4.4 version of sqoop. Could you please suggest what code is stable and we can make that as base of our development. And Regarding the feature list, I will send you a separate mail with list of features. A couple of them are very generic and we are planning to contribute them back to community. May be we will need your help for that. Thanks and Regards, Rakesh. From: Atul Gupta <[email protected]<mailto:[email protected]>> Date: Tuesday, September 23, 2014 at 12:42 PM To: Abraham Elmahrek <[email protected]<mailto:[email protected]>> Cc: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>>, Shashank Tandon <[email protected]<mailto:[email protected]>>, Rakesh Sharma <[email protected]<mailto:[email protected]>> Subject: Re: Stable release for sqoop ++rakesh Regards, Atul On 23-Sep-2014, at 12:16 pm, Abraham Elmahrek <[email protected]<mailto:[email protected]>> wrote: Hey there, Could you provide your incremental import command? Also, what security enhancements and customizations are you looking into? It might be something that fits generically into Sqoop. Sqoop 1.4.5 is the latest community release of Sqoop. -Abe On Mon, Sep 22, 2014 at 9:59 PM, Rakesh Sharma <[email protected]<mailto:[email protected]>> wrote: Hi, We are planning to use Sqoop at Expedia but we might have to do some enhancements and customizations into Sqoop. We were planning to base our work with 1.4.5 branch but we found a couple of issues with code from that branch. eg., We were not able to import data from mysql db using incremental import with 1.4.5 branch but it is running properly with latest code and even in 1.4.4 branch as well. Could you please suggest the stable version after 1.4.5 which we can pick to use as base for our development. Thanks and Regards, Rakesh.
