> On July 13, 2015, 3:18 p.m., Jarek Cecho wrote: > > Overall the patch looks good. I have only one concern - I've tried to load > > generated avro file into Hive and it seems that it doesn't work. Here is > > what I've did: > > > > 1) Create MySQL table > > > > CREATE TABLE `decimals` ( > > `id` int(11) NOT NULL AUTO_INCREMENT, > > `dec1` decimal(3,2) DEFAULT NULL, > > PRIMARY KEY (`id`) > > ) ENGINE=MyISAM AUTO_INCREMENT=3 DEFAULT CHARSET=latin1; > > > > 2) Insert some data into it > > > > insert into decimals values(2, 1.00); > > insert into decimals values(1, 0.04); > > insert into decimals values(3, 5.10); > > insert into decimals values(4, 3.33); > > > > 3) Import data with Sqoop > > > > sqoop import --connect jdbc:mysql://mysql/sqoop --username sqoop --password > > sqoop --table decimals --as-avrodatafile > > > > 4) Manually create corresponding table in Hive > > > > create table decimals(id int, dec decimal(3,2)) stored as avro; > > > > 5) Load data > > > > load data inpath '/user/root/decimals' overwrite into table decimals; > > > > 6) And finally verify data: > > > > hive> select * from decimals; > > OK > > 2 NULL > > 1 NULL > > 3 NULL > > 4 NULL > > Time taken: 0.45 seconds, Fetched: 4 row(s) > > > > I would expect to see the proper decimal values, but I'm getting NULLs > > instead. I'm wondering whether I'm doing something wrong or whether we do > > have a small bug somewhere (that might not necessarily be on the Sqoop > > side).
Hmm Hive and Impala might not support "fixed" data types with logical type "decimal". Good catch. I'll change this to bytes. - Abraham ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35859/#review91470 ----------------------------------------------------------- On July 11, 2015, 12:28 a.m., Abraham Elmahrek wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/35859/ > ----------------------------------------------------------- > > (Updated July 11, 2015, 12:28 a.m.) > > > Review request for Sqoop. > > > Bugs: SQOOP-1493 > https://issues.apache.org/jira/browse/SQOOP-1493 > > > Repository: sqoop-trunk > > > Description > ------- > > commit 8583077a33956cd2a3abc05f73172210c31994a9 > Author: Abraham Elmahrek <[email protected]> > Date: Fri Jun 19 05:45:25 2015 +0300 > > SQOOP-1493: Add ability to import/export true decimal in Avro instead of > serializing it to String > > :100644 100644 2e3d884... 628491c... M ivy/libraries.properties > :100644 100644 ee3cf62... 5970981... M > src/java/org/apache/sqoop/avro/AvroUtil.java > :100644 100644 d9569c5... 5650079... M > src/java/org/apache/sqoop/manager/ConnManager.java > :100644 100644 20f056a... 76c3458... M > src/java/org/apache/sqoop/mapreduce/AvroExportMapper.java > :100644 100644 aed1e72... a4ac46e... M > src/java/org/apache/sqoop/mapreduce/AvroOutputFormat.java > :100644 100644 ab263c1... b60ee42... M > src/java/org/apache/sqoop/mapreduce/GenericRecordExportMapper > :100644 100644 2576673... 8490582... M > src/java/org/apache/sqoop/orm/AvroSchemaGenerator.java > :100644 100644 1c6f7f4... aff8d08... M > src/java/org/apache/sqoop/orm/ClassWriter.java > :100644 100644 663828c... 42e36ab... M > src/test/com/cloudera/sqoop/TestAvroExport.java > :100644 100644 260e80a... 2f680c8... M > src/test/com/cloudera/sqoop/TestAvroImport.java > > > Diffs > ----- > > LICENSE.txt c36c7ad > ivy/libraries.properties 2e3d884 > src/java/org/apache/sqoop/avro/AvroUtil.java ee3cf62 > src/java/org/apache/sqoop/config/ConfigurationConstants.java e19c17b > src/java/org/apache/sqoop/manager/ConnManager.java d9569c5 > src/java/org/apache/sqoop/mapreduce/AvroExportMapper.java 20f056a > src/java/org/apache/sqoop/mapreduce/AvroImportMapper.java 0ea5ca4 > src/java/org/apache/sqoop/mapreduce/AvroOutputFormat.java aed1e72 > src/java/org/apache/sqoop/mapreduce/GenericRecordExportMapper.java ab263c1 > src/java/org/apache/sqoop/orm/AvroSchemaGenerator.java 2576673 > src/java/org/apache/sqoop/orm/ClassWriter.java 1c6f7f4 > src/test/com/cloudera/sqoop/TestAvroExport.java 663828c > src/test/com/cloudera/sqoop/TestAvroImport.java 260e80a > src/test/org/apache/sqoop/TestAvroImport.java PRE-CREATION > > Diff: https://reviews.apache.org/r/35859/diff/ > > > Testing > ------- > > ant test && manual > > > Thanks, > > Abraham Elmahrek > >
