> On Aug. 20, 2014, 5:46 a.m., Abraham Elmahrek wrote: > > connector/connector-generic-hdfs/pom.xml, line 31 > > <https://reviews.apache.org/r/24878/diff/1/?file=664959#file664959line31> > > > > sqoop-connector-hdfs?
Yeah, I wasn't 100% sure. My thought is that there will be additional HDFS-based connectors (I had Kite in mind) and they can inherit from this one. But yeah, nothing actually makes it particularly "generic", so I'll rename. > On Aug. 20, 2014, 5:46 a.m., Abraham Elmahrek wrote: > > connector/connector-generic-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/GenericHdfsDestroyer.java, > > line 33 > > <https://reviews.apache.org/r/24878/diff/1/?file=664963#file664963line33> > > > > Follow up Jira for sure! A "done" flag is super useful for integration > > with Oozie. https://issues.apache.org/jira/browse/SQOOP-1457 > On Aug. 20, 2014, 5:46 a.m., Abraham Elmahrek wrote: > > connector/connector-generic-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/GenericHdfsInitializer.java, > > line 43 > > <https://reviews.apache.org/r/24878/diff/1/?file=664965#file664965line43> > > > > Follow up Jira to define some kind of Schema? Can expect CSV data and > > fields are Strings at the minimum. The framework should be able to handle > > coercing types! SQOOP-1458 > On Aug. 20, 2014, 5:46 a.m., Abraham Elmahrek wrote: > > connector/connector-generic-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/ConnectionForm.java, > > line 28 > > <https://reviews.apache.org/r/24878/diff/1/?file=664971#file664971line28> > > > > Follow up Jira: > > > > Hostname, port, scheme, or some mixture of every thing? For example: > > > > hdfs://notmylocalhost:8050 SQOOP-1459 - Gwen ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/24878/#review51070 ----------------------------------------------------------- On Aug. 20, 2014, 2:03 a.m., Gwen Shapira wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/24878/ > ----------------------------------------------------------- > > (Updated Aug. 20, 2014, 2:03 a.m.) > > > Review request for Sqoop. > > > Repository: sqoop-SQOOP-1367 > > > Description > ------- > > Refactored framework code into an HDFS connector > > > Diffs > ----- > > client/src/main/java/org/apache/sqoop/client/SqoopClient.java 1d93ae3 > common/pom.xml 9bfa07d > common/src/main/java/org/apache/sqoop/common/PrefixContext.java > PRE-CREATION > connector/connector-generic-hdfs/pom.xml PRE-CREATION > > connector/connector-generic-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/GenericHdfsConnector.java > PRE-CREATION > > connector/connector-generic-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/GenericHdfsConnectorError.java > PRE-CREATION > > connector/connector-generic-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/GenericHdfsConstants.java > PRE-CREATION > > connector/connector-generic-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/GenericHdfsDestroyer.java > PRE-CREATION > > connector/connector-generic-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/GenericHdfsExtractor.java > PRE-CREATION > > connector/connector-generic-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/GenericHdfsInitializer.java > PRE-CREATION > > connector/connector-generic-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/GenericHdfsLoader.java > PRE-CREATION > > connector/connector-generic-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/GenericHdfsPartition.java > PRE-CREATION > > connector/connector-generic-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/GenericHdfsPartitioner.java > PRE-CREATION > > connector/connector-generic-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/GenericHdfsValidator.java > PRE-CREATION > > connector/connector-generic-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/ConnectionConfiguration.java > PRE-CREATION > > connector/connector-generic-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/ConnectionForm.java > PRE-CREATION > > connector/connector-generic-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/FromJobConfiguration.java > PRE-CREATION > > connector/connector-generic-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/InputForm.java > PRE-CREATION > > connector/connector-generic-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/OutputCompression.java > PRE-CREATION > > connector/connector-generic-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/OutputForm.java > PRE-CREATION > > connector/connector-generic-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/OutputFormat.java > PRE-CREATION > > connector/connector-generic-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/StorageType.java > PRE-CREATION > > connector/connector-generic-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/ToJobConfiguration.java > PRE-CREATION > > connector/connector-generic-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/hdfsWriter/HdfsSequenceWriter.java > PRE-CREATION > > connector/connector-generic-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/hdfsWriter/HdfsTextWriter.java > PRE-CREATION > > connector/connector-generic-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/hdfsWriter/genericHdfsWriter.java > PRE-CREATION > > connector/connector-generic-hdfs/src/main/resources/generic-hdfs-connector-resources.properties > PRE-CREATION > > connector/connector-generic-hdfs/src/main/resources/sqoopconnector.properties > PRE-CREATION > > connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/CSVIntermediateDataFormat.java > 39d48c7 > > connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/IntermediateDataFormatError.java > 9219074 > connector/pom.xml d642c3e > core/src/main/java/org/apache/sqoop/framework/FrameworkValidator.java > f19a23e > core/src/main/java/org/apache/sqoop/framework/JobManager.java 48360ea > core/src/main/java/org/apache/sqoop/framework/SubmissionRequest.java > be6099e > > core/src/main/java/org/apache/sqoop/framework/configuration/ExportJobConfiguration.java > 6665429 > > core/src/main/java/org/apache/sqoop/framework/configuration/ImportJobConfiguration.java > 2a35eb9 > core/src/main/java/org/apache/sqoop/framework/configuration/InputForm.java > d5cbeec > > core/src/main/java/org/apache/sqoop/framework/configuration/JobConfiguration.java > 7c653bf > > core/src/main/java/org/apache/sqoop/framework/configuration/OutputCompression.java > 6cac46d > core/src/main/java/org/apache/sqoop/framework/configuration/OutputForm.java > b2cdb44 > > core/src/main/java/org/apache/sqoop/framework/configuration/OutputFormat.java > 4cd3589 > > core/src/main/java/org/apache/sqoop/framework/configuration/StorageType.java > dbe9f95 > core/src/test/java/org/apache/sqoop/framework/TestFrameworkValidator.java > f875ceb > core/src/test/java/org/apache/sqoop/repository/TestJdbcRepository.java > 31df04c > > execution/mapreduce/src/main/java/org/apache/sqoop/execution/mapreduce/MapreduceExecutionEngine.java > ff328cb > execution/mapreduce/src/main/java/org/apache/sqoop/job/PrefixContext.java > c3beed7 > > execution/mapreduce/src/main/java/org/apache/sqoop/job/etl/HdfsExportExtractor.java > 27afd8c > > execution/mapreduce/src/main/java/org/apache/sqoop/job/etl/HdfsExportPartition.java > cdbdaa8 > > execution/mapreduce/src/main/java/org/apache/sqoop/job/etl/HdfsExportPartitioner.java > b3590dc > > execution/mapreduce/src/main/java/org/apache/sqoop/job/etl/HdfsSequenceImportLoader.java > d4ffb13 > > execution/mapreduce/src/main/java/org/apache/sqoop/job/etl/HdfsTextImportLoader.java > 7b799ca > > execution/mapreduce/src/main/java/org/apache/sqoop/job/mr/SqoopDestroyerExecutor.java > 8e31ef5 > > execution/mapreduce/src/main/java/org/apache/sqoop/job/mr/SqoopInputFormat.java > e96909a > execution/mapreduce/src/main/java/org/apache/sqoop/job/mr/SqoopMapper.java > 6e2cfbf > > execution/mapreduce/src/main/java/org/apache/sqoop/job/mr/SqoopOutputFormatLoadExecutor.java > 01c32e4 > > execution/mapreduce/src/test/java/org/apache/sqoop/execution/mapreduce/MapreduceExecutionEngineTest.java > 0f2a882 > execution/mapreduce/src/test/java/org/apache/sqoop/job/TestHdfsExtract.java > e460c3e > execution/mapreduce/src/test/java/org/apache/sqoop/job/TestHdfsLoad.java > 65e82b1 > pom.xml 38b4974 > > repository/repository-derby/src/main/java/org/apache/sqoop/repository/derby/DerbyRepositoryHandler.java > f355ceb > server/pom.xml dc89409 > server/src/main/java/org/apache/sqoop/handler/JobRequestHandler.java > 6f2df93 > test/pom.xml 8001fce > test/src/main/java/org/apache/sqoop/test/testcases/ConnectorTestCase.java > 5ec4fa4 > > test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/TableImportTest.java > 9e6f991 > > test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/imports/PartitionerTest.java > 1bc3b93 > > test/src/test/java/org/apache/sqoop/integration/server/SubmissionWithDisabledModelObjectsTest.java > 126ca32 > > Diff: https://reviews.apache.org/r/24878/diff/ > > > Testing > ------- > > From MySQL to HDFS uncompressed text > and from HDFS uncompressed text to MySQL > > > Thanks, > > Gwen Shapira > >
