> On Sept. 26, 2017, 1:35 p.m., Szabolcs Vasas wrote: > > Hi Sandish, > > > > Thank you for your patch! Please find my findings below: > > > > - Sqoop uses Kite for importing in Parquet format so I think your patch > > should also utilize the Kite libraries for Hive table generation (see > > org.apache.sqoop.mapreduce.ParquetJob#createDataset). This would ensure > > that the table creation behaviour is the same when we import the data and > > when we only create the table. > > - SQLServerHiveImportTest.testNormalHiveImportAsAvro test case fails > > (SQLServerHiveImportTest is a subclass of TestHiveImport so it has > > inherited your new test case). Can you please take a look? > > > > I have not had time yet to deeply review the Avro part but my impression is > > that it needs much more thorough testing(e.g. more data types) since it is > > a new feature. What do you think? > > > > Regards, > > Szabolcs > > Sandish Kumar HN wrote: > - Do you want me to use > org.apache.sqoop.mapreduce.ParquetJob#createDataset in testcases?? > - Yes will add more data types on testcases for avro/parquet
I haven't checked if ParquetJob#createDataset is a perfect fit for your use case here but looks like a good starting point. - Szabolcs ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/61882/#review186246 ----------------------------------------------------------- On Aug. 24, 2017, 9:51 a.m., Sandish Kumar HN wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/61882/ > ----------------------------------------------------------- > > (Updated Aug. 24, 2017, 9:51 a.m.) > > > Review request for Sqoop and Anna Szonyi. > > > Bugs: SQOOP-3215 > https://issues.apache.org/jira/browse/SQOOP-3215 > > > Repository: sqoop-trunk > > > Description > ------- > > sqoop import and create hive table as formats(avro,parquet) > > > Diffs > ----- > > src/java/org/apache/sqoop/hive/TableDefWriter.java deec32d6 > src/java/org/apache/sqoop/tool/BaseSqoopTool.java 1564bdcb > src/test/com/cloudera/sqoop/hive/TestHiveImport.java a624f52b > testdata/hive/scripts/createhiveImportasparquet.q PRE-CREATION > testdata/hive/scripts/normalHiveImportAvro.q PRE-CREATION > > > Diff: https://reviews.apache.org/r/61882/diff/1/ > > > Testing > ------- > > testNormalHiveImportAsAvro, testNormalHiveImportAsParquet with table creation > checks. > > > Thanks, > > Sandish Kumar HN > >
