----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/28794/#review65025 -----------------------------------------------------------
I didn't finished reviewing the patch entirely, but wanted at least share couple of observations that I've made so far :) Just FYI, the patch doesn't apply cleanly any more, there is small conflict in connector/pom.xml because of Kafka connector. common-test/src/main/java/org/apache/sqoop/common/test/db/DatabaseProvider.java <https://reviews.apache.org/r/28794/#comment107951> Nit: The comments seems copy&pasted from different method? :) connector/connector-mysql-dump/pom.xml <https://reviews.apache.org/r/28794/#comment107952> Hmmm... so we have one connector depending on the other. That is undesirable as now the JDBC Connector is quite locked. I would much rather pull "shared" functionality to SDK then have inter-connector dependencies. connector/connector-mysql-dump/src/main/java/org/apache/sqoop/connector/mysqldump/MySQLDumpConnectorUtils.java <https://reviews.apache.org/r/28794/#comment107953> It would be great if the temporary directory would be inside the map tasks working directory. We're not currently exposing location of working directory in our APIs, so I would assume that it's not that easy. Perhaps we can create follow up JIRA to expose temporary directory that connector can use in our APIs? Jarcec - Jarek Cecho On Dec. 10, 2014, 4:52 a.m., Abraham Elmahrek wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/28794/ > ----------------------------------------------------------- > > (Updated Dec. 10, 2014, 4:52 a.m.) > > > Review request for Sqoop. > > > Bugs: SQOOP-1859 > https://issues.apache.org/jira/browse/SQOOP-1859 > > > Repository: sqoop-sqoop2 > > > Description > ------- > > commit 34e79a259f1dad2fcce3c394a403f552f02cf79e > Author: Abraham Elmahrek <[email protected]> > Date: Sun Dec 7 15:15:07 2014 -0600 > > SQOOP-1859: Sqoop2: MySQL Fast Connector > > :100644 100644 9814ac8... 8af0f8f... M > common-test/src/main/java/org/apache/sqoop/common/test/db/MySQLProvider.java > :100644 100644 b68d8d0... d0697b7... M > connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcPartitioner.java > :100644 100644 11a6bca... a98c7ee... M > connector/connector-mysql-fastpath/pom.xml > :000000 100644 0000000... 7da3232... A > connector/connector-mysql-fastpath/src/main/java/org/apache/sqoop/connector/mysqlfast/BasicMySQLDumpProcessor.java > :000000 100644 0000000... 7950146... A > connector/connector-mysql-fastpath/src/main/java/org/apache/sqoop/connector/mysqlfast/MySQLDumpProcessor.java > :000000 100644 0000000... 02bd801... A > connector/connector-mysql-fastpath/src/main/java/org/apache/sqoop/connector/mysqlfast/MySQLFastConnector.java > :000000 100644 0000000... 5d74bf0... A > connector/connector-mysql-fastpath/src/main/java/org/apache/sqoop/connector/mysqlfast/MySQLFastConnectorConstants.java > :000000 100644 0000000... 354d236... A > connector/connector-mysql-fastpath/src/main/java/org/apache/sqoop/connector/mysqlfast/MySQLFastConnectorError.java > :000000 100644 0000000... 9168330... A > connector/connector-mysql-fastpath/src/main/java/org/apache/sqoop/connector/mysqlfast/MySQLFastConnectorUpgrader.java > :000000 100644 0000000... 0e6a3b1... A > connector/connector-mysql-fastpath/src/main/java/org/apache/sqoop/connector/mysqlfast/MySQLFastExtractor.java > :000000 100644 0000000... 2ec65fe... A > connector/connector-mysql-fastpath/src/main/java/org/apache/sqoop/connector/mysqlfast/MySQLFastFromDestroyer.java > :000000 100644 0000000... b3296b6... A > connector/connector-mysql-fastpath/src/main/java/org/apache/sqoop/connector/mysqlfast/MySQLFastFromInitializer.java > :000000 100644 0000000... 50eb6d8... A > connector/connector-mysql-fastpath/src/main/java/org/apache/sqoop/connector/mysqlfast/MySQLFastLoader.java > :000000 100644 0000000... 9ca01e3... A > connector/connector-mysql-fastpath/src/main/java/org/apache/sqoop/connector/mysqlfast/MySQLFastPartitioner.java > :000000 100644 0000000... af72194... A > connector/connector-mysql-fastpath/src/main/java/org/apache/sqoop/connector/mysqlfast/MySQLFastToDestroyer.java > :000000 100644 0000000... 21f9650... A > connector/connector-mysql-fastpath/src/main/java/org/apache/sqoop/connector/mysqlfast/MySQLFastToInitializer.java > :000000 100644 0000000... bc6cf24... A > connector/connector-mysql-fastpath/src/main/java/org/apache/sqoop/connector/mysqlfast/NamedFifo.java > :000000 100644 0000000... c9aac61... A > connector/connector-mysql-fastpath/src/main/java/org/apache/sqoop/connector/mysqlfast/configuration/FromJobConfig.java > :000000 100644 0000000... 48584a4... A > connector/connector-mysql-fastpath/src/main/java/org/apache/sqoop/connector/mysqlfast/configuration/FromJobConfiguration.java > :000000 100644 0000000... 1bf3960... A > connector/connector-mysql-fastpath/src/main/java/org/apache/sqoop/connector/mysqlfast/configuration/LinkConfig.java > :000000 100644 0000000... 6c8d770... A > connector/connector-mysql-fastpath/src/main/java/org/apache/sqoop/connector/mysqlfast/configuration/LinkConfiguration.java > :000000 100644 0000000... a615544... A > connector/connector-mysql-fastpath/src/main/java/org/apache/sqoop/connector/mysqlfast/configuration/ToJobConfig.java > :000000 100644 0000000... d874fd1... A > connector/connector-mysql-fastpath/src/main/java/org/apache/sqoop/connector/mysqlfast/configuration/ToJobConfiguration.java > :000000 100644 0000000... 42df6b8... A > connector/connector-mysql-fastpath/src/main/resources/mysql-fast-connector-config.properties > :000000 100644 0000000... d7b95a2... A > connector/connector-mysql-fastpath/src/main/resources/sqoopconnector.properties > :000000 100644 0000000... 36cd153... A > connector/connector-mysql-fastpath/src/test/java/org/apache/sqoop/connector/mysqlfast/TestBasicMySQLDumpProcessor.java > :000000 100644 0000000... e4376e5... A > connector/connector-mysql-fastpath/src/test/java/org/apache/sqoop/connector/mysqlfast/TestMySQLFastConnectorExtractor.java > :000000 100644 0000000... e3cdea3... A > connector/connector-mysql-fastpath/src/test/java/org/apache/sqoop/connector/mysqlfast/TestMySQLFastConnectorLoader.java > :000000 100644 0000000... 44ffced... A > connector/connector-mysql-fastpath/src/test/resources/log4j.properties > :100644 100644 26a35f5... 41c7644... M > connector/connector-mysql-jdbc/src/main/resources/sqoopconnector.properties > :100644 100644 da4ed3e... 016da52... M connector/pom.xml > :100644 100644 e182176... 09a4bb9... M pom.xml > :100644 100644 1adcca0... 3879c87... M server/pom.xml > :100644 100644 eedb545... e11a844... M test/pom.xml > > > Diffs > ----- > > > common-test/src/main/java/org/apache/sqoop/common/test/db/DatabaseProvider.java > 0a12d7b > connector/connector-mysql-dump/pom.xml PRE-CREATION > > connector/connector-mysql-dump/src/main/java/org/apache/sqoop/connector/mysqldump/BasicMySQLDumpProcessor.java > PRE-CREATION > > connector/connector-mysql-dump/src/main/java/org/apache/sqoop/connector/mysqldump/MySQLDumpConnector.java > PRE-CREATION > > connector/connector-mysql-dump/src/main/java/org/apache/sqoop/connector/mysqldump/MySQLDumpConnectorConstants.java > PRE-CREATION > > connector/connector-mysql-dump/src/main/java/org/apache/sqoop/connector/mysqldump/MySQLDumpConnectorError.java > PRE-CREATION > > connector/connector-mysql-dump/src/main/java/org/apache/sqoop/connector/mysqldump/MySQLDumpConnectorUpgrader.java > PRE-CREATION > > connector/connector-mysql-dump/src/main/java/org/apache/sqoop/connector/mysqldump/MySQLDumpConnectorUtils.java > PRE-CREATION > > connector/connector-mysql-dump/src/main/java/org/apache/sqoop/connector/mysqldump/MySQLDumpExtractor.java > PRE-CREATION > > connector/connector-mysql-dump/src/main/java/org/apache/sqoop/connector/mysqldump/MySQLDumpFromDestroyer.java > PRE-CREATION > > connector/connector-mysql-dump/src/main/java/org/apache/sqoop/connector/mysqldump/MySQLDumpFromInitializer.java > PRE-CREATION > > connector/connector-mysql-dump/src/main/java/org/apache/sqoop/connector/mysqldump/MySQLDumpLoader.java > PRE-CREATION > > connector/connector-mysql-dump/src/main/java/org/apache/sqoop/connector/mysqldump/MySQLDumpPartitioner.java > PRE-CREATION > > connector/connector-mysql-dump/src/main/java/org/apache/sqoop/connector/mysqldump/MySQLDumpProcessBuilder.java > PRE-CREATION > > connector/connector-mysql-dump/src/main/java/org/apache/sqoop/connector/mysqldump/MySQLDumpProcessor.java > PRE-CREATION > > connector/connector-mysql-dump/src/main/java/org/apache/sqoop/connector/mysqldump/MySQLDumpToDestroyer.java > PRE-CREATION > > connector/connector-mysql-dump/src/main/java/org/apache/sqoop/connector/mysqldump/MySQLDumpToInitializer.java > PRE-CREATION > > connector/connector-mysql-dump/src/main/java/org/apache/sqoop/connector/mysqldump/MySQLImportProcessBuilder.java > PRE-CREATION > > connector/connector-mysql-dump/src/main/java/org/apache/sqoop/connector/mysqldump/configuration/FromJobConfig.java > PRE-CREATION > > connector/connector-mysql-dump/src/main/java/org/apache/sqoop/connector/mysqldump/configuration/FromJobConfiguration.java > PRE-CREATION > > connector/connector-mysql-dump/src/main/java/org/apache/sqoop/connector/mysqldump/configuration/LinkConfig.java > PRE-CREATION > > connector/connector-mysql-dump/src/main/java/org/apache/sqoop/connector/mysqldump/configuration/LinkConfiguration.java > PRE-CREATION > > connector/connector-mysql-dump/src/main/java/org/apache/sqoop/connector/mysqldump/configuration/ToJobConfig.java > PRE-CREATION > > connector/connector-mysql-dump/src/main/java/org/apache/sqoop/connector/mysqldump/configuration/ToJobConfiguration.java > PRE-CREATION > > connector/connector-mysql-dump/src/main/resources/mysql-dump-connector-config.properties > PRE-CREATION > connector/connector-mysql-dump/src/main/resources/sqoopconnector.properties > PRE-CREATION > > connector/connector-mysql-dump/src/test/java/org/apache/sqoop/connector/mysqldump/TestBasicMySQLDumpProcessor.java > PRE-CREATION > > connector/connector-mysql-dump/src/test/java/org/apache/sqoop/connector/mysqldump/TestLinkConfiguration.java > PRE-CREATION > > connector/connector-mysql-dump/src/test/java/org/apache/sqoop/connector/mysqldump/TestMySQLDumpConnectorExtractor.java > PRE-CREATION > > connector/connector-mysql-dump/src/test/java/org/apache/sqoop/connector/mysqldump/TestMySQLDumpConnectorLoader.java > PRE-CREATION > > connector/connector-mysql-dump/src/test/java/org/apache/sqoop/connector/mysqldump/TestMySQLDumpProcessBuilder.java > PRE-CREATION > > connector/connector-mysql-dump/src/test/java/org/apache/sqoop/connector/mysqldump/TestMySQLImportProcessBuilder.java > PRE-CREATION > connector/connector-mysql-dump/src/test/resources/log4j.properties > PRE-CREATION > connector/connector-mysql-fastpath/pom.xml 11a6bca > connector/connector-mysql-jdbc/src/main/resources/sqoopconnector.properties > 26a35f5 > connector/pom.xml da4ed3e > pom.xml e182176 > server/pom.xml 1adcca0 > test/pom.xml eedb545 > > test/src/main/java/org/apache/sqoop/test/minicluster/TomcatSqoopMiniCluster.java > 4aa8562 > > Diff: https://reviews.apache.org/r/28794/diff/ > > > Testing > ------- > > Ran unit tests. Still need to do manual tests. > > > Thanks, > > Abraham Elmahrek > >
