----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/14085/#review26039 -----------------------------------------------------------
Thanks for working on this Shuaishuai Nie. Can you add some unit tests (this option being recognized properly for example). Also, documentation can be updated to describe this feature - Venkat Ranganathan On Sept. 11, 2013, 5:12 p.m., Shuaishuai Nie wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/14085/ > ----------------------------------------------------------- > > (Updated Sept. 11, 2013, 5:12 p.m.) > > > Review request for Sqoop. > > > Bugs: SQOOP-1192 > https://issues.apache.org/jira/browse/SQOOP-1192 > > > Repository: sqoop-trunk > > > Description > ------- > > Now Sqoop will copy jar files in %SQOOP_HOME%\lib folder to the job cache > every time a Sqoop job is launched. When Oozie launch a Sqoop job, this > behavior can be optimized by add these jars in Oozie Sqoop sharelib. In this > case, the jar files in share lib only needed be localized to each worker node > once and reuse by all Sqoop job launched by Oozie. This can reduce massive > disk I/O on worker node when using Sqoop by Oozie. To enable this, Sqoop need > to have an option which enable the job to skip adding lib jars to the job > cache. For now, this option should only be used by Oozie started Sqoop job. > The patch attached introduce "--skip-dist-cache" option to enable this > feature. > > > Diffs > ----- > > src/java/org/apache/sqoop/SqoopOptions.java 01805f9 > src/java/org/apache/sqoop/mapreduce/JobBase.java 322df1c > src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatUtilities.java b05f587 > src/java/org/apache/sqoop/tool/BaseSqoopTool.java ebb1857 > > Diff: https://reviews.apache.org/r/14085/diff/ > > > Testing > ------- > > Tested the new option with Oozie-Sqoop workflow to ensure it doesn't break > Sqoop library dependencies when launched by Oozie > > > Thanks, > > Shuaishuai Nie > >
