> On July 10, 2013, 6:21 a.m., Hari Shreedharan wrote: > > execution/mapreduce/src/main/java/org/apache/sqoop/job/mr/SqoopInputFormat.java, > > line 69 > > <https://reviews.apache.org/r/12261/diff/2/?file=319204#file319204line69> > > > > Shouldn't the schema be passed to the partitioner too? Maybe useful for > > some connectors to create partitions.
That should definitely be the case, thank you for pointing that out Hari! - Jarek ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/12261/#review22948 ----------------------------------------------------------- On July 10, 2013, 3:05 p.m., Jarek Cecho wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/12261/ > ----------------------------------------------------------- > > (Updated July 10, 2013, 3:05 p.m.) > > > Review request for Sqoop. > > > Bugs: SQOOP-1121 > https://issues.apache.org/jira/browse/SQOOP-1121 > > > Repository: sqoop-sqoop2 > > > Description > ------- > > The patch seems big, but it's just touching a lot of files. Couple of high > level notes: > > * I've added serialization of the schema into mapreduce job > * I've refactored a bit ConfigurationUtils by providing better encapsulation > and adding unit tests for that class. > * I've added the schema to all MR context objects (that are passed to > extractor, loader, ...) > * Currently only Connector schema is being used everywhere, I'm expecting to > change that once Framework (=hio) schema will be in place. > > > Diffs > ----- > > common/src/main/java/org/apache/sqoop/job/etl/DestroyerContext.java > 10cfb103b97a367c87c9ab264bf83035a5547660 > common/src/main/java/org/apache/sqoop/job/etl/ExtractorContext.java > f9d7a8b57b999db4531e46ce62e2ac535e580e3f > common/src/main/java/org/apache/sqoop/job/etl/LoaderContext.java > dad19f15672375084d3c9d6736c19628853222c9 > common/src/main/java/org/apache/sqoop/job/etl/PartitionerContext.java > 5e7cea75cb5d683c4bdc47674a7b6ecc56881b32 > > connector/connector-generic-jdbc/src/test/java/org/apache/sqoop/connector/jdbc/TestExportLoader.java > 50a32d91f3a6f2290a6519754de9427e167df20b > > connector/connector-generic-jdbc/src/test/java/org/apache/sqoop/connector/jdbc/TestImportExtractor.java > 54ffe5ba44a12ca71b08cd93a9ed347255cb8d25 > > connector/connector-generic-jdbc/src/test/java/org/apache/sqoop/connector/jdbc/TestImportPartitioner.java > 0afec49f1b46d2b0c8b30a029507f5de6f7e7ee6 > core/src/main/java/org/apache/sqoop/framework/JobManager.java > 58d6c104bbc589f33857f43f8831448635758ce8 > execution/mapreduce/pom.xml 31da5f11887415db85e26c6ccb2500383d9b2635 > execution/mapreduce/src/main/java/org/apache/sqoop/job/JobConstants.java > e2b3ce84484c8a5a984a8b31a4d83e5ca03c0ba4 > > execution/mapreduce/src/main/java/org/apache/sqoop/job/mr/ConfigurationUtils.java > 64ec4375fd4c2edb48f605b4997cccd29d420a22 > > execution/mapreduce/src/main/java/org/apache/sqoop/job/mr/SqoopDestroyerExecutor.java > 4493a45f720c015d285030e5b7b1382e1f3f16e7 > > execution/mapreduce/src/main/java/org/apache/sqoop/job/mr/SqoopInputFormat.java > 0721b7e20f00c0d8559356919ca79bbdcbe669b5 > execution/mapreduce/src/main/java/org/apache/sqoop/job/mr/SqoopMapper.java > 7715d5f7be23c838d337f2c117af330843e99e8f > > execution/mapreduce/src/main/java/org/apache/sqoop/job/mr/SqoopOutputFormatLoadExecutor.java > d47f861035a9eb199d166dc9f54542654b63a9b8 > execution/mapreduce/src/test/java/org/apache/sqoop/job/TestHdfsExtract.java > 58c306824ca307eb062e6664cb1962072cb5526b > execution/mapreduce/src/test/java/org/apache/sqoop/job/TestHdfsLoad.java > b9b063193d39b005a1c36bf7a5399559c39ac925 > execution/mapreduce/src/test/java/org/apache/sqoop/job/TestMapReduce.java > ee03427ba0ca7009c0ea2edf15e4f482f6584ed2 > > execution/mapreduce/src/test/java/org/apache/sqoop/job/mr/TestConfigurationUtils.java > PRE-CREATION > > execution/mapreduce/src/test/java/org/apache/sqoop/job/mr/TestSqoopOutputFormatLoadExecutor.java > d2b501eb56ae0708754047150e0618e1f6470395 > > submission/mapreduce/src/main/java/org/apache/sqoop/submission/mapreduce/MapreduceSubmissionEngine.java > 0e8c9f72339f1f6d82ce2a7c49cc1645369f480d > > Diff: https://reviews.apache.org/r/12261/diff/ > > > Testing > ------- > > Provided new unit test for the util class ConfigurationUtils. Both unit and > integration tests seems to be working, plus I've tested the changes on real > cluster. > > > Thanks, > > Jarek Cecho > >
