Hmm... I've just marked problem with default AVRO mapping of type DATE as "Won't fix" in SQOOP-399 because it would break backward compatibility, but I'm starting to think that it needs to be addressed anyway. I'll look on it later this week. Can you please file JIRA for that?
We've recently added option to force sqoop use different than default mapping, however version 1.4.0 do not support that for AVRO output. But there is working patch attached to SQOOP-362 that has been committed to the trunk and will be included in next release. Meantime you might consider applying this patch to the base 1.4.0 release and rebuilding sqoop for you. Can you also file JIRA for your second issue? You can also submit your patch for review in case that you managed to fix that issue. Thank you very much for your time during investigating the issues! Jarcec SQOOP-399 https://issues.apache.org/jira/browse/SQOOP-399 SQOOP-362 https://issues.apache.org/jira/browse/SQOOP-362 On Wed, Dec 07, 2011 at 11:48:24AM +0100, Antoine Pieretti wrote: > > Hi Kate, > > here the versions : > Sqoop : 1.4.0 > Oraoop : oraoop-1.4.0.149 > Hadoop : hadoop-core-0.20.2-cdh3u2 > > The first thing we found, is that the AvroSchemaGenerator, return a LONG > type for a DATE in toAvroType, but ORAOOP always works with String. > > Please, find here the console log during an Update > (See attached file: SQOOP_UPDATE_LOG-txt.zip) > > In debug, I found that in TextExportMapper, he call parse method, wich > __loadFromFields with a List<String>, with the first value equals to the > text "Objavro.schema?{"type":"record". > > If in the JdbcUpdateExportJob I implement the AvroExportMapper for an Avro > fyleType, it works. > > Thanks for your help. > > Please, also find an avro file example. > > > (See attached file: part-m-00000-avro.zip) > > Regards, Antoine > > > > > > > > Kate Ting > <k...@cloudera.co > m> A > sqoop-user@incubator.apache.org > 07/12/2011 09:27 cc > > Objet > Veuillez répondre Re: SQOOP AVRO ORAOOP some > à questions > sqoop-user@incuba > tor.apache.org > > > > > > > > > Hi Antoine, > > The following would be helpful in figuring out what could be causing the > failed job: > > What Sqoop, OraOop, and Hadoop versions are you running? > Please run Sqoop job with --verbose flag and then attach console log. > Also, provide a representative input data set that triggers this problem. > > Regards, Kate > > On Tue, Dec 6, 2011 at 12:15 AM, Antoine Pieretti < > anpieretti-...@airfrance.fr> wrote: > > > Hi folks, > > i am a newbie within Sqoop and Avro. > > I try to manipulate data from an Oracle Database, import, and export > Avro files. > > The import works well, the export (INSERT) also, but when i am trying > to Update data from an Avro file to the DB the job failed. > > Here my command line : export --connect jdbc:oracle:thin:@qxxxxxxx > --username xxxxxxxxxxxxxxxxx_O --password xxxxxxxxxxx_O --table > MY_TABLE --verbose --export-dir D:\\tmp\extraction\Rep --update-key > My_COLUMN_KEY. > > The job calls the TextExportMapper (not the AvropExportMapper), and > try to parse the file, wich contains the Avro schemas and parse it as > a data line. > > Is it a common issue ? > > Thanks for your help > > Antoine PIERETTI > > > > Accédez aux meilleurs tarifs Air France, gérez vos réservations et > enregistrez-vous en ligne sur http://www.airfrance.com > Find best Air France fares, manage your reservations and check in > online at http://www.airfrance.com > > > Les données et renseignements contenus dans ce message peuvent être > de nature confidentielle et soumis au secret professionnel et sont > destinés à l'usage exclusif du destinataire dont les coordonnées > figurent ci-dessus. Si vous recevez cette communication par erreur, > nous vous demandons de ne pas la copier, l'utiliser ou la divulguer. > Nous vous prions de notifier cette erreur à l'expéditeur et d'effacer > immédiatement cette communication de votre système. Société Air > France - Société anonyme au capital de 1 901 231 625 euros - RCS > Bobigny (France) 420 495 178 - 45, rue de Paris, 95 747 Roissy CDG > CEDEX > The data and information contained in this message may be > confidential and subject to professional secrecy and are intended for > the exclusive use of the recipient at the address shown above. If you > receive this message by mistake, we ask you not to copy, use or > disclose it. Please notify this error to the sender immediately and > delete this message from your system. Société Air France - Limited > company with capital of 1,901,231,625 euros - Bobigny register of > companies (France) 420 495 178 - 45, rue de Paris, 95 747 Roissy CDG > CEDEX Pensez à l'environnement avant d'imprimer ce message. > Think of the environment before printing this mail. > > > > > > > Accédez aux meilleurs tarifs Air France, gérez vos réservations et > enregistrez-vous en ligne sur http://www.airfrance.com > Find best Air France fares, manage your reservations and check in online at > http://www.airfrance.com Les données et renseignements contenus dans ce > message peuvent être de nature confidentielle et soumis au secret > professionnel et sont destinés à l'usage exclusif du destinataire dont les > coordonnées figurent ci-dessus. Si vous recevez cette communication par > erreur, nous vous demandons de ne pas la copier, l'utiliser ou la divulguer. > Nous vous prions de notifier cette erreur à l'expéditeur et d'effacer > immédiatement cette communication de votre système. Société Air France - > Société anonyme au capital de 1 901 231 625 euros - RCS Bobigny (France) 420 > 495 178 - 45, rue de Paris, 95 747 Roissy CDG CEDEX > The data and information contained in this message may be confidential and > subject to professional secrecy and are intended for the exclusive use of the > recipient at the address shown above. If you receive this message by mistake, > we ask you not to copy, use or disclose it. Please notify this error to the > sender immediately and delete this message from your system. Société Air > France - Limited company with capital of 1,901,231,625 euros - Bobigny > register of companies (France) 420 495 178 - 45, rue de Paris, 95 747 Roissy > CDG CEDEX Pensez à l'environnement avant d'imprimer ce message. > Think of the environment before printing this mail.
signature.asc
Description: Digital signature
