Sorry for jumping on this late. What is the version of mysql server and mysql java connector you are using. I think some older versions of server and client (5.1.17 or earlier of client and some similar older versions of server) exhibit the problem.
Thanks Venkat On Thu, Sep 25, 2014 at 8:48 PM, Rakesh Sharma <[email protected]> wrote: > Hi Abe, > > We have tried it with different tables but had same issue. For your > reference, the table details are below : > > Hive table : > ========= > hive> show create table test_employee; > OK > CREATE EXTERNAL TABLE `test_employee`( > `id` int, > `name` string, > `salary` int, > `country` string, > `last_updated` timestamp) > ROW FORMAT SERDE > 'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe' > STORED AS INPUTFORMAT > 'org.apache.hadoop.mapred.TextInputFormat' > OUTPUTFORMAT > 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat' > LOCATION > 'hdfs://sandbox.hortonworks.com:8020/data/hive-ext/Test_Employee' > TBLPROPERTIES ( > 'numFiles'='0', > 'transient_lastDdlTime'='1411566509', > 'COLUMN_STATS_ACCURATE'='false', > 'totalSize'='0', > 'numRows'='-1', > 'rawDataSize'='-1') > Time taken: 0.7 seconds, Fetched: 21 row(s) > > Mysql Table: > ========== > show create table Test_Employee; > > +---------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ > | Table | Create Table > > > > | > > +---------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ > | Test_Employee | CREATE TABLE `Test_Employee` ( > `id` int(11) DEFAULT NULL, > `name` varchar(20) DEFAULT NULL, > `salary` int(11) DEFAULT NULL, > `country` varchar(20) DEFAULT NULL, > `LAST_UPDATED` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE > CURRENT_TIMESTAMP > ) ENGINE=MyISAM DEFAULT CHARSET=latin1 | > > +---------------+————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————+ > > Any help will be highly appreciated. > > Thanks and Regards, > Rakesh. > > > From: Abraham Elmahrek <[email protected]<mailto:[email protected]>> > Date: Friday, September 26, 2014 at 5:25 AM > To: Rakesh Sharma <[email protected]<mailto:[email protected]>> > Cc: Atul Gupta <[email protected]<mailto:[email protected]>>, " > [email protected]<mailto:[email protected]>" <[email protected] > <mailto:[email protected]>>, Shashank Tandon <[email protected] > <mailto:[email protected]>> > Subject: Re: Stable release for sqoop > > I'd still recommend Sqoop 1.4.5 or use a version of Sqoop that's provided > by one of the vendors. > > For reference though, could you provide the Schema of this table? > > On Thu, Sep 25, 2014 at 3:41 AM, Rakesh Sharma <[email protected] > <mailto:[email protected]>> wrote: > Hi Abe, > > Thanks for the prompt reply and support. We are using the following sqoop > command for code from 1.4.5 branch: > > sqoop import -Dmapred.job.queue.name<http://Dmapred.job.queue.name>=default > --connect jdbc:mysql://localhost/test --username root --password '' --table > Temp_Employee --incremental lastmodified --check-column last_updated > --last-value "2014-001-01 00:00:00" --hive-import > --hive-table="Temp_Employee" -m 1 > > And we are getting the following exception : > 14/09/25 03:35:11 INFO manager.SqlManager: Executing SQL statement: SELECT > t.* FROM `Temp_Employee` AS t LIMIT 1 > 14/09/25 03:35:11 ERROR manager.SqlManager: Error reading from database: > java.sql.SQLException: Streaming result set > com.mysql.jdbc.RowDataDynamic@69222fc3 is still active. No statements may > be issued when any streaming result sets are open and in use on a given > connection. Ensure that you have called .close() on any active streaming > result sets before attempting more queries. > java.sql.SQLException: Streaming result set > com.mysql.jdbc.RowDataDynamic@69222fc3 is still active. No statements may > be issued when any streaming result sets are open and in use on a given > connection. Ensure that you have called .close() on any active streaming > result sets before attempting more queries. > at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:934) > at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:931) > at > com.mysql.jdbc.MysqlIO.checkForOutstandingStreamingData(MysqlIO.java:2735) > at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1899) > at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2151) > at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2619) > at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2569) > at com.mysql.jdbc.StatementImpl.executeQuery(StatementImpl.java:1524) > at > com.mysql.jdbc.ConnectionImpl.getMaxBytesPerChar(ConnectionImpl.java:3003) > at com.mysql.jdbc.Field.getMaxBytesPerCharacter(Field.java:602) > at > com.mysql.jdbc.ResultSetMetaData.getPrecision(ResultSetMetaData.java:445) > at > org.apache.sqoop.manager.SqlManager.getColumnInfoForRawQuery(SqlManager.java:285) > at > org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:240) > at org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:226) > at > org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:295) > at org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1773) > at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1578) > at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:96) > at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:478) > at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:601) > at org.apache.sqoop.Sqoop.run(Sqoop.java:143) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179) > at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218) > at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227) > at org.apache.sqoop.Sqoop.main(Sqoop.java:236) > 14/09/25 03:35:11 ERROR tool.ImportTool: Encountered IOException running > import job: java.io.IOException: No columns to generate for ClassWriter > at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1584) > at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:96) > at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:478) > at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:601) > at org.apache.sqoop.Sqoop.run(Sqoop.java:143) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179) > at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218) > at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227) > at org.apache.sqoop.Sqoop.main(Sqoop.java:236) > > The same command works on 1.4.6(trunk) and even on 1.4.4 version of sqoop. > Could you please suggest what code is stable and we can make that as base > of our development. > > And Regarding the feature list, I will send you a separate mail with list > of features. A couple of them are very generic and we are planning to > contribute them back to community. May be we will need your help for that. > > Thanks and Regards, > Rakesh. > > > > From: Atul Gupta <[email protected]<mailto:[email protected]>> > Date: Tuesday, September 23, 2014 at 12:42 PM > To: Abraham Elmahrek <[email protected]<mailto:[email protected]>> > Cc: "[email protected]<mailto:[email protected]>" < > [email protected]<mailto:[email protected]>>, Shashank Tandon < > [email protected]<mailto:[email protected]>>, Rakesh Sharma < > [email protected]<mailto:[email protected]>> > Subject: Re: Stable release for sqoop > > ++rakesh > > Regards, > Atul > > On 23-Sep-2014, at 12:16 pm, Abraham Elmahrek <[email protected]<mailto: > [email protected]>> wrote: > > Hey there, > > Could you provide your incremental import command? Also, what security > enhancements and customizations are you looking into? It might be something > that fits generically into Sqoop. Sqoop 1.4.5 is the latest community > release of Sqoop. > > -Abe > > On Mon, Sep 22, 2014 at 9:59 PM, Rakesh Sharma <[email protected] > <mailto:[email protected]>> wrote: > Hi, > > We are planning to use Sqoop at Expedia but we might have to do some > enhancements and customizations into Sqoop. We were planning to base our > work with 1.4.5 branch but we found a couple of issues with code from that > branch. > eg., We were not able to import data from mysql db using incremental > import with 1.4.5 branch but it is running properly with latest code and > even in 1.4.4 branch as well. > > Could you please suggest the stable version after 1.4.5 which we can pick > to use as base for our development. > > Thanks and Regards, > Rakesh. > > > > > > -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
