----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/24878/#review51070 -----------------------------------------------------------
Good stuff Gwen! A few comments below, but in general +1 from me! common/pom.xml <https://reviews.apache.org/r/24878/#comment89021> This seems similar to the parent pom definition. You should be able to remove the activation configuration since it exists in the parent pom. common/pom.xml <https://reviews.apache.org/r/24878/#comment89022> This seems similar to the parent pom definition. You should be able to remove the activation configuration since it exists in the parent pom. common/pom.xml <https://reviews.apache.org/r/24878/#comment89023> Not necessary here? connector/connector-generic-hdfs/pom.xml <https://reviews.apache.org/r/24878/#comment89025> sqoop-connector-hdfs? connector/connector-generic-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/GenericHdfsConnector.java <https://reviews.apache.org/r/24878/#comment89026> Should never hit this case... but I feel like we should throw an exception. connector/connector-generic-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/GenericHdfsDestroyer.java <https://reviews.apache.org/r/24878/#comment89027> Follow up Jira for sure! A "done" flag is super useful for integration with Oozie. connector/connector-generic-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/GenericHdfsInitializer.java <https://reviews.apache.org/r/24878/#comment89029> Follow up Jira to define some kind of Schema? Can expect CSV data and fields are Strings at the minimum. The framework should be able to handle coercing types! connector/connector-generic-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/GenericHdfsLoader.java <https://reviews.apache.org/r/24878/#comment89031> rm -rf? connector/connector-generic-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/GenericHdfsLoader.java <https://reviews.apache.org/r/24878/#comment89030> rm -rf connector/connector-generic-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/ConnectionForm.java <https://reviews.apache.org/r/24878/#comment89032> Follow up Jira: Hostname, port, scheme, or some mixture of every thing? For example: hdfs://notmylocalhost:8050 connector/pom.xml <https://reviews.apache.org/r/24878/#comment89024> Yes! core/src/main/java/org/apache/sqoop/framework/FrameworkValidator.java <https://reviews.apache.org/r/24878/#comment89013> Nit: Kill off some of the whitespace here. core/src/main/java/org/apache/sqoop/framework/SubmissionRequest.java <https://reviews.apache.org/r/24878/#comment89015> Nit: Spacing seems weird here. core/src/main/java/org/apache/sqoop/framework/SubmissionRequest.java <https://reviews.apache.org/r/24878/#comment89016> Nit: Kill space. execution/mapreduce/src/test/java/org/apache/sqoop/execution/mapreduce/MapreduceExecutionEngineTest.java <https://reviews.apache.org/r/24878/#comment89017> Nit: Tab oddness. server/src/main/java/org/apache/sqoop/handler/JobRequestHandler.java <https://reviews.apache.org/r/24878/#comment89018> Nit: Spacing. - Abraham Elmahrek On Aug. 20, 2014, 2:03 a.m., Gwen Shapira wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/24878/ > ----------------------------------------------------------- > > (Updated Aug. 20, 2014, 2:03 a.m.) > > > Review request for Sqoop. > > > Repository: sqoop-SQOOP-1367 > > > Description > ------- > > Refactored framework code into an HDFS connector > > > Diffs > ----- > > client/src/main/java/org/apache/sqoop/client/SqoopClient.java 1d93ae3 > common/pom.xml 9bfa07d > common/src/main/java/org/apache/sqoop/common/PrefixContext.java > PRE-CREATION > connector/connector-generic-hdfs/pom.xml PRE-CREATION > > connector/connector-generic-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/GenericHdfsConnector.java > PRE-CREATION > > connector/connector-generic-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/GenericHdfsConnectorError.java > PRE-CREATION > > connector/connector-generic-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/GenericHdfsConstants.java > PRE-CREATION > > connector/connector-generic-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/GenericHdfsDestroyer.java > PRE-CREATION > > connector/connector-generic-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/GenericHdfsExtractor.java > PRE-CREATION > > connector/connector-generic-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/GenericHdfsInitializer.java > PRE-CREATION > > connector/connector-generic-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/GenericHdfsLoader.java > PRE-CREATION > > connector/connector-generic-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/GenericHdfsPartition.java > PRE-CREATION > > connector/connector-generic-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/GenericHdfsPartitioner.java > PRE-CREATION > > connector/connector-generic-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/GenericHdfsValidator.java > PRE-CREATION > > connector/connector-generic-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/ConnectionConfiguration.java > PRE-CREATION > > connector/connector-generic-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/ConnectionForm.java > PRE-CREATION > > connector/connector-generic-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/FromJobConfiguration.java > PRE-CREATION > > connector/connector-generic-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/InputForm.java > PRE-CREATION > > connector/connector-generic-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/OutputCompression.java > PRE-CREATION > > connector/connector-generic-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/OutputForm.java > PRE-CREATION > > connector/connector-generic-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/OutputFormat.java > PRE-CREATION > > connector/connector-generic-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/StorageType.java > PRE-CREATION > > connector/connector-generic-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/ToJobConfiguration.java > PRE-CREATION > > connector/connector-generic-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/hdfsWriter/HdfsSequenceWriter.java > PRE-CREATION > > connector/connector-generic-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/hdfsWriter/HdfsTextWriter.java > PRE-CREATION > > connector/connector-generic-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/hdfsWriter/genericHdfsWriter.java > PRE-CREATION > > connector/connector-generic-hdfs/src/main/resources/generic-hdfs-connector-resources.properties > PRE-CREATION > > connector/connector-generic-hdfs/src/main/resources/sqoopconnector.properties > PRE-CREATION > > connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/CSVIntermediateDataFormat.java > 39d48c7 > > connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/IntermediateDataFormatError.java > 9219074 > connector/pom.xml d642c3e > core/src/main/java/org/apache/sqoop/framework/FrameworkValidator.java > f19a23e > core/src/main/java/org/apache/sqoop/framework/JobManager.java 48360ea > core/src/main/java/org/apache/sqoop/framework/SubmissionRequest.java > be6099e > > core/src/main/java/org/apache/sqoop/framework/configuration/ExportJobConfiguration.java > 6665429 > > core/src/main/java/org/apache/sqoop/framework/configuration/ImportJobConfiguration.java > 2a35eb9 > core/src/main/java/org/apache/sqoop/framework/configuration/InputForm.java > d5cbeec > > core/src/main/java/org/apache/sqoop/framework/configuration/JobConfiguration.java > 7c653bf > > core/src/main/java/org/apache/sqoop/framework/configuration/OutputCompression.java > 6cac46d > core/src/main/java/org/apache/sqoop/framework/configuration/OutputForm.java > b2cdb44 > > core/src/main/java/org/apache/sqoop/framework/configuration/OutputFormat.java > 4cd3589 > > core/src/main/java/org/apache/sqoop/framework/configuration/StorageType.java > dbe9f95 > core/src/test/java/org/apache/sqoop/framework/TestFrameworkValidator.java > f875ceb > core/src/test/java/org/apache/sqoop/repository/TestJdbcRepository.java > 31df04c > > execution/mapreduce/src/main/java/org/apache/sqoop/execution/mapreduce/MapreduceExecutionEngine.java > ff328cb > execution/mapreduce/src/main/java/org/apache/sqoop/job/PrefixContext.java > c3beed7 > > execution/mapreduce/src/main/java/org/apache/sqoop/job/etl/HdfsExportExtractor.java > 27afd8c > > execution/mapreduce/src/main/java/org/apache/sqoop/job/etl/HdfsExportPartition.java > cdbdaa8 > > execution/mapreduce/src/main/java/org/apache/sqoop/job/etl/HdfsExportPartitioner.java > b3590dc > > execution/mapreduce/src/main/java/org/apache/sqoop/job/etl/HdfsSequenceImportLoader.java > d4ffb13 > > execution/mapreduce/src/main/java/org/apache/sqoop/job/etl/HdfsTextImportLoader.java > 7b799ca > > execution/mapreduce/src/main/java/org/apache/sqoop/job/mr/SqoopDestroyerExecutor.java > 8e31ef5 > > execution/mapreduce/src/main/java/org/apache/sqoop/job/mr/SqoopInputFormat.java > e96909a > execution/mapreduce/src/main/java/org/apache/sqoop/job/mr/SqoopMapper.java > 6e2cfbf > > execution/mapreduce/src/main/java/org/apache/sqoop/job/mr/SqoopOutputFormatLoadExecutor.java > 01c32e4 > > execution/mapreduce/src/test/java/org/apache/sqoop/execution/mapreduce/MapreduceExecutionEngineTest.java > 0f2a882 > execution/mapreduce/src/test/java/org/apache/sqoop/job/TestHdfsExtract.java > e460c3e > execution/mapreduce/src/test/java/org/apache/sqoop/job/TestHdfsLoad.java > 65e82b1 > pom.xml 38b4974 > > repository/repository-derby/src/main/java/org/apache/sqoop/repository/derby/DerbyRepositoryHandler.java > f355ceb > server/pom.xml dc89409 > server/src/main/java/org/apache/sqoop/handler/JobRequestHandler.java > 6f2df93 > test/pom.xml 8001fce > test/src/main/java/org/apache/sqoop/test/testcases/ConnectorTestCase.java > 5ec4fa4 > > test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/TableImportTest.java > 9e6f991 > > test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/imports/PartitionerTest.java > 1bc3b93 > > test/src/test/java/org/apache/sqoop/integration/server/SubmissionWithDisabledModelObjectsTest.java > 126ca32 > > Diff: https://reviews.apache.org/r/24878/diff/ > > > Testing > ------- > > From MySQL to HDFS uncompressed text > and from HDFS uncompressed text to MySQL > > > Thanks, > > Gwen Shapira > >
