> On July 26, 2012, 9:13 p.m., Jarek Cecho wrote: > > /src/java/org/apache/sqoop/mapreduce/PGBulkloadExportReducer.java, line 83 > > <https://reviews.apache.org/r/2724/diff/3/?file=129307#file129307line83> > > > > I'm thinking whether it would be beneficial here not to die > > immediately. For example if user is exporting 30 partitions and there will > > be some issues with the first. He would be forced to manually do all 30 of > > them. But if we would try all them first and log only the failed ones, user > > would have to fix only that one. What do you think sir?
The motivation for the current design is keeping destination table clean by staging in a single atomic transaction. I think making a number of reduce tasks configurable is a good solution. Users can choose single atomic staging or parallel staging. - Masatake ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2724/#review9514 ----------------------------------------------------------- On July 26, 2012, 10:41 a.m., Masatake Iwasaki wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/2724/ > ----------------------------------------------------------- > > (Updated July 26, 2012, 10:41 a.m.) > > > Review request for Sqoop. > > > Description > ------- > > Patch for SQOOP-390 > https://issues.apache.org/jira/browse/SQOOP-390 > > > This addresses bug SQOOP-390. > https://issues.apache.org/jira/browse/SQOOP-390 > > > Diffs > ----- > > /src/java/org/apache/sqoop/manager/PGBulkloadManager.java PRE-CREATION > /src/java/org/apache/sqoop/mapreduce/AutoProgressReducer.java PRE-CREATION > /src/java/org/apache/sqoop/mapreduce/PGBulkloadExportJob.java PRE-CREATION > /src/java/org/apache/sqoop/mapreduce/PGBulkloadExportMapper.java > PRE-CREATION > /src/java/org/apache/sqoop/mapreduce/PGBulkloadExportReducer.java > PRE-CREATION > /src/test/com/cloudera/sqoop/manager/PGBulkloadManagerManualTest.java > PRE-CREATION > > Diff: https://reviews.apache.org/r/2724/diff/ > > > Testing > ------- > > This patch include the test class PGBulkloadManagerTest. > I've tested "ant test" and passed. > > > Thanks, > > Masatake Iwasaki > >
