----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62492/#review205364 -----------------------------------------------------------
Hi Chris, Since this is a quite big patch I will review it iteratively. In the first iteration I would like you to move the mainframe-related code to the mainframe classes since your changes affect the mainframe connector only, please see my comments below. I have also seen many unused imports added by your changes, please remove those. I have started to add separate comment for every unused import but the list is not complete please check the other classes too. src/java/org/apache/sqoop/mapreduce/DataDrivenImportJob.java Lines 40 (patched) <https://reviews.apache.org/r/62492/#comment288261> Unused import. src/java/org/apache/sqoop/mapreduce/DataDrivenImportJob.java Lines 43 (patched) <https://reviews.apache.org/r/62492/#comment288262> Unnecessary new line. src/java/org/apache/sqoop/mapreduce/DataDrivenImportJob.java Lines 55 (patched) <https://reviews.apache.org/r/62492/#comment288263> Unused import. src/java/org/apache/sqoop/mapreduce/DataDrivenImportJob.java Lines 57 (patched) <https://reviews.apache.org/r/62492/#comment288264> Unused import. src/java/org/apache/sqoop/mapreduce/DataDrivenImportJob.java Lines 58 (patched) <https://reviews.apache.org/r/62492/#comment288265> Unused import. src/java/org/apache/sqoop/mapreduce/DataDrivenImportJob.java Lines 59 (patched) <https://reviews.apache.org/r/62492/#comment288266> Unused import. src/java/org/apache/sqoop/mapreduce/DataDrivenImportJob.java Line 91 (original), 100 (patched) <https://reviews.apache.org/r/62492/#comment288268> Since the BinaryFile layout is only supported by the mainframe connector we should move this branch to org.apache.sqoop.mapreduce.mainframe.MainframeImportJob#configureMapper. src/java/org/apache/sqoop/mapreduce/DataDrivenImportJob.java Lines 205 (patched) <https://reviews.apache.org/r/62492/#comment288269> Since the BinaryFile layout is only supported by the mainframe connector we should move this branch to org.apache.sqoop.mapreduce.mainframe.MainframeImportJob#getOutputFormatClass. src/java/org/apache/sqoop/mapreduce/RawKeyTextOutputFormat.java Lines 70 (patched) <https://reviews.apache.org/r/62492/#comment288270> This behavior should be moved to another class since it is really unexpected in a class called RawKeyTextOutputFormat. I think you should introduce a new class for mainframe output format and if the format is text then you just delegate to an instance of RawKeyTextOutputFormat otherwise you return an instance of BinaryKeyRecordWriter. - Szabolcs Vasas On June 18, 2018, 1:47 a.m., Chris Teoh wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/62492/ > ----------------------------------------------------------- > > (Updated June 18, 2018, 1:47 a.m.) > > > Review request for Sqoop. > > > Bugs: SQOOP-3224 > https://issues.apache.org/jira/browse/SQOOP-3224 > > > Repository: sqoop-trunk > > > Description > ------- > > Added --as-binaryfile and --buffersize to support FTP transfer mode switching. > > > Diffs > ----- > > build.xml 0ae729bc > src/docs/user/import-mainframe.txt abeb7cde > src/java/org/apache/sqoop/SqoopOptions.java d9984af3 > src/java/org/apache/sqoop/mapreduce/DataDrivenImportJob.java 3b542102 > src/java/org/apache/sqoop/mapreduce/KeyRecordWriters.java PRE-CREATION > src/java/org/apache/sqoop/mapreduce/RawKeyTextOutputFormat.java fec34f21 > > src/java/org/apache/sqoop/mapreduce/mainframe/AbstractMainframeDatasetImportMapper.java > PRE-CREATION > src/java/org/apache/sqoop/mapreduce/mainframe/MainframeConfiguration.java > ea54b07f > > src/java/org/apache/sqoop/mapreduce/mainframe/MainframeDatasetBinaryImportMapper.java > PRE-CREATION > > src/java/org/apache/sqoop/mapreduce/mainframe/MainframeDatasetBinaryRecord.java > PRE-CREATION > > src/java/org/apache/sqoop/mapreduce/mainframe/MainframeDatasetFTPRecordReader.java > 1f78384b > > src/java/org/apache/sqoop/mapreduce/mainframe/MainframeDatasetImportMapper.java > 0b7b5b85 > src/java/org/apache/sqoop/mapreduce/mainframe/MainframeImportJob.java > 8ef30d38 > src/java/org/apache/sqoop/tool/BaseSqoopTool.java c62ee98c > src/java/org/apache/sqoop/tool/ImportTool.java 2c474b7e > src/java/org/apache/sqoop/tool/MainframeImportTool.java 8883301d > src/java/org/apache/sqoop/util/MainframeFTPClientUtils.java 95bc0ecb > src/test/org/apache/sqoop/manager/mainframe/MainframeManagerImportTest.java > 041dfb78 > src/test/org/apache/sqoop/manager/mainframe/MainframeTestUtil.java f28ff36c > > src/test/org/apache/sqoop/mapreduce/mainframe/TestMainframeDatasetBinaryRecord.java > PRE-CREATION > > src/test/org/apache/sqoop/mapreduce/mainframe/TestMainframeDatasetFTPRecordReader.java > 3547294f > src/test/org/apache/sqoop/tool/TestMainframeImportTool.java 0b0c6c34 > src/test/org/apache/sqoop/util/TestMainframeFTPClientUtils.java 90a85194 > > > Diff: https://reviews.apache.org/r/62492/diff/17/ > > > Testing > ------- > > Unit tests. > > Functional testing on mainframe. > > > Thanks, > > Chris Teoh > >
