Glad that you were able to workaround it. Are you open to share the whole map task log?
Jarcec > On Oct 5, 2015, at 10:28 AM, Timothy Garza <[email protected]> > wrote: > > The logs don't give away anything different than the Exception shown below: > >> Error: java.io.IOException: Can't export data, please check failed map task >> logs >> at >> org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:112) >> at >> org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:39) >> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:152) >> at >> org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64) >> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:773) >> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) >> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:175) >> at java.security.AccessController.doPrivileged(Native Method) >> at javax.security.auth.Subject.doAs(Subject.java:415) >> at >> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548) >> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:170) >> >> Caused by: java.lang.RuntimeException: Can't parse input data: <data row >> displayed here in clear text>’ >> at products.__loadFromFields(products.java:421) >> at products.parse(products.java:344) >> at >> org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java: >> 83) >> > > BUT - I have now been successful when I use the comma delimiter (I've been > using pipe '|'). I will dig deeper into the Delimiter library to see what > options are available as a comma-delimiter is bad for our data. > > Kind Regards > > > Timothy Garza > Database and BI Developer > Collinson Technology Services > Skype: timothy.garza.cts > collinsongroup.com > > > > > -----Original Message----- > From: Jarek Jarcec Cecho [mailto:[email protected]] On Behalf Of Jarek Jarcec > Cecho > Sent: 05 October 2015 17:21 > To: [email protected] > Subject: Re: SQOOP export of Sequence Files (Hive tables) > > > Hi Timothy, > please check my feedback inline: > >> Error: java.io.IOException: Can't export data, please check failed map >> task logs > > Can you please share the failed map task log? It should contain significantly > more information about the failed state. Usually this means that the data are > corrupted (Sqoop will fail on first error whereas Hive will simply ignore > that, hence people don’t always realize that the data are corrupted) or that > the structure between Hive and target database is different/ > >> I’ve discovered through trial & error that SQOOP 1.4.6 can export SEQUENCE >> files which have been created by SQOOP but it cannot read SEQUENCE files >> created by Hive. > > That is correct. Sqoop is not able to read Hive’s sequence file. We have that > note for import in the user guide [1] but it’s relevant for export as well. > > Jarcec > > Links: > 1: > http://sqoop.apache.org/docs/1.4.6/SqoopUserGuide.html#_importing_data_into_hive > >> On Oct 5, 2015, at 9:02 AM, Timothy Garza <[email protected]> >> wrote: >> >> I’ve upgraded to Hive v1.2.1 (from v0.11 and v0.13) and I’m still unable to >> output even Hive textfiles to MySQL using SQOOP. Has anybody got any advice >> for getting Hive/Impala data out of HDFS and into a RDBMS? >> >> I successfully configured Hive v1.2.1, then created a new Hive table ( >> CREATE TABLE (<columns>,… STORED AS TEXTFILE ). >> >> sqoop export \ >> --connect jdbc:mysql://<URL>:<port>/<databasename> \ --username >> <username> \ --password <password> \ --table products \ --export-dir >> 'hdfs://<masterIP>:9000/<hdfs path to file> >> >> Error: java.io.IOException: Can't export data, please check failed map task >> logs >> at >> org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:112) >> at >> org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:39) >> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:152) >> at >> org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64) >> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:773) >> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) >> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:175) >> at java.security.AccessController.doPrivileged(Native Method) >> at javax.security.auth.Subject.doAs(Subject.java:415) >> at >> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548) >> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:170) >> >> Caused by: java.lang.RuntimeException: Can't parse input data: <data row >> displayed here in clear text>’ >> at products.__loadFromFields(products.java:421) >> at products.parse(products.java:344) >> at >> org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java: >> 83) >> >> >> From: Timothy Garza >> Sent: 11 September 2015 11:24 >> To: [email protected] >> Subject: RE: SQOOP export of Sequence Files (Hive tables) >> >> Updates on this thread: >> >> I’ve discovered through trial & error that SQOOP 1.4.6 can export SEQUENCE >> files which have been created by SQOOP but it cannot read SEQUENCE files >> created by Hive. >> >> >> >> >> >> From: Timothy Garza >> Sent: 03 September 2015 23:25 >> To: [email protected] >> Subject: SQOOP export of Sequence Files (Hive tables) >> >> Can we confirm or deny that Sqoop 1.4.6 can or can't export Sequence files >> to RDBMS via JDBC driver. >> >> I get an error from Sqoop regarding not being able to cast BytesWriteable to >> LongWriteable regardless of which data-type i'm reading from (in a Hive >> table) or writing to (MySQL or SQL Server table). >> >> There is no mention in the documentation of being able to export FROM (or >> not being able to export FROM) a Sequence File stored in HDFS. >> >> I can successfully SQOOP data from a TEXTFILE stored in HDFS but this sort >> of takes away the point of SQOOP and HDFS (in my opinion). >> >> Can anyone clarify this? >> >> >> >> The Collinson Group Limited; Registered number: 2577557, Registered in >> England & Wales; Registered Office: Cutlers Exchange, 123 Houndsditch, >> London, EC3A 7BU. >> >> >> This e-mail may contain privileged and confidential information and/or >> copyright material and is intended for the use of the addressee only. If you >> receive this e-mail by mistake please advise the sender immediately by using >> the reply facility in your e-mail software and delete this e-mail from your >> computer system. You may not deliver, copy or disclose its contents to >> anyone else. Any unauthorised use may be unlawful. Any views expressed in >> this e-mail are those of the individual sender and may not necessarily >> reflect the views of The Collinson Group Ltd and/or its subsidiaries or any >> other associated company (collectively “Collinson Group”). >> >> As communications via the Internet are not secure Collinson Group cannot >> accept any liability if this e-mail is accessed by third parties during the >> course of transmission or is modified or amended in any way following >> despatch. Collinson Group cannot guarantee that any attachment to this email >> does not contain a virus, therefore it is strongly recommended that you >> carry out your own virus check before opening any attachment, as we cannot >> accept liability for any damage sustained as a result of software virus >> infection. Senders of messages shall be taken to consent to the monitoring >> and recording of e-mails addressed to members of the Company. >
