----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/23531/#review48183 -----------------------------------------------------------
client/src/main/java/org/apache/sqoop/client/SqoopClient.java <https://reviews.apache.org/r/23531/#comment84535> Shouldn't job just need connection id? Associated connection is already driving "from" and "to" connectors, correct? common/src/main/java/org/apache/sqoop/common/ConnectorType.java <https://reviews.apache.org/r/23531/#comment84536> Can you please fill in the javadocs for this enum? The name seems to be suggesting that we have two types of connectors, "FROM" and "TO", but that is not the case, right? One connector can (and most will) support both "FROM" and "TO" and hence we should rename this enum? common/src/main/java/org/apache/sqoop/json/FrameworkBean.java <https://reviews.apache.org/r/23531/#comment84537> Why do we still need FrameworkBean, should it be removed? - Jarek Cecho On July 18, 2014, 10:33 p.m., Abraham Elmahrek wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/23531/ > ----------------------------------------------------------- > > (Updated July 18, 2014, 10:33 p.m.) > > > Review request for Sqoop, Gwen Shapira, Hari Shreedharan, and Jarek Cecho. > > > Bugs: SQOOP-1376 > https://issues.apache.org/jira/browse/SQOOP-1376 > > > Repository: sqoop-sqoop2 > > > Description > ------- > > commit 8327ff3b6432f2d9358991b694945411fff26025 > Author: Abraham Elmahrek <[email protected]> > Date: Wed Jul 9 12:42:17 2014 -0700 > > Sqoop2: From/To: Refactor connector interface > > Various components have been hacked together to compile. > Their working order are to be addressed in follow up Jiras. > This includes, but is not limited to: > Repository, Execution Engine, Submission Engine, Core, > Server, and the shell. > > ConectorBean and JobBean have a simple design. Improvements > are to be addressed in follow up Jiras. > > MConnector now has 2 MJobForms instances. > There is a MJobForms instance for the "from" connector and > a MJobForms instance for the "to" connector. > MConnector still has only one MConnectionForms. > The information needed when extracting using one connector may > be different than the information needed when loading using > the same connector. Example: GenericJdbcConnector can use > a staging table when it is the connector being used to load > data TO. It does not need a staging table when extracting data > FROM. > Since Connections define how Sqoop connects to a database, > there shouldn't be any variation based on FROM/TO. > > MJob objects must now be aware of Connectors that it is > extracting data FROM and sending data TO. It also keeps > track of Connection objects for both Connectors. > > SPI changed to allow FROM/TO. Initializers, laoders, etc. > were not changed, just the classes that extend > CallbackBase. > > :100644 100644 05ea6d6... d1f7a1d... M > client/src/main/java/org/apache/sqoop/client/SqoopClient.java > :100644 100644 cbe049a... f6b2de4... M > common/src/main/java/org/apache/sqoop/json/ConnectorBean.java > :100644 100644 eb79f98... e234908... M > common/src/main/java/org/apache/sqoop/json/FrameworkBean.java > :100644 100644 1555bd5... 58c53d3... M > common/src/main/java/org/apache/sqoop/json/JobBean.java > :000000 100644 0000000... c9a1c42... A > common/src/main/java/org/apache/sqoop/json/JobValidationBean.java > :100644 100644 43fad27... 117ecdc... M > common/src/main/java/org/apache/sqoop/model/MConnector.java > :100644 100644 c742459... 580db9c... M > common/src/main/java/org/apache/sqoop/model/MFramework.java > :100644 100644 849168d... 155f246... M > common/src/main/java/org/apache/sqoop/model/MJob.java > :100644 100644 f697023... 8505483... M > common/src/main/java/org/apache/sqoop/model/MJobForms.java > :100644 100644 e0da80f... ab33568... M > connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcConnector.java > :100644 100644 0c5f6e1... 3e2f89a... M > connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcValidator.java > :100644 100644 17215f0... 346b625... M > connector/connector-mysql-jdbc/src/main/java/org/apache/sqoop/connector/mysqljdbc/MySqlJdbcConnector.java > :100644 100644 b80de7f... 9cb6244... M > core/src/main/java/org/apache/sqoop/connector/ConnectorHandler.java > :100644 100644 505121c... fe21a3f... M > core/src/main/java/org/apache/sqoop/framework/FrameworkManager.java > :100644 100644 f5f6a36... 0bd4947... M > core/src/main/java/org/apache/sqoop/framework/FrameworkValidator.java > :100644 100644 e052584... b5bd3a2... M > core/src/main/java/org/apache/sqoop/framework/JobManager.java > :100644 100644 53d0039... 9e8c07c... M > core/src/main/java/org/apache/sqoop/framework/SubmissionRequest.java > :000000 100644 0000000... 7c653bf... A > core/src/main/java/org/apache/sqoop/framework/configuration/JobConfiguration.java > :100644 100644 ecf5004... 95a8bd5... M > core/src/main/java/org/apache/sqoop/repository/Repository.java > :100644 100644 5c0a027... 6352bb5... M > execution/mapreduce/src/main/java/org/apache/sqoop/execution/mapreduce/MapreduceExecutionEngine.java > :100644 100644 bd11323... 96ecf32... M > execution/mapreduce/src/main/java/org/apache/sqoop/job/mr/ConfigurationUtils.java > :100644 100644 e1a95a7... 0edcb7a... M > execution/mapreduce/src/main/java/org/apache/sqoop/job/mr/SqoopDestroyerExecutor.java > :100644 100644 6891258... 516c80a... M > execution/mapreduce/src/main/java/org/apache/sqoop/job/mr/SqoopInputFormat.java > :100644 100644 92de37e... 37f416d... M > execution/mapreduce/src/main/java/org/apache/sqoop/job/mr/SqoopMapper.java > :100644 100644 7dedee9... b5adace... M > execution/mapreduce/src/main/java/org/apache/sqoop/job/mr/SqoopOutputFormatLoadExecutor.java > :100644 100644 5bce3a9... f002db6... M > repository/repository-derby/src/main/java/org/apache/sqoop/repository/derby/DerbyRepositoryHandler.java > :100644 100644 fcbb475... 1a77360... M > repository/repository-derby/src/main/java/org/apache/sqoop/repository/derby/DerbySchemaConstants.java > :100644 100644 7042a53... fa16360... M > repository/repository-derby/src/main/java/org/apache/sqoop/repository/derby/DerbySchemaQuery.java > :100644 100644 362ba79... a9394b6... M > server/src/main/java/org/apache/sqoop/handler/JobRequestHandler.java > :100644 100644 9cce17b... 99edb6b... M > shell/src/main/java/org/apache/sqoop/shell/CloneConnectionFunction.java > :100644 100644 f80552c... 11878f1... M > shell/src/main/java/org/apache/sqoop/shell/CloneJobFunction.java > :100644 100644 598adbc... e063f9a... M > shell/src/main/java/org/apache/sqoop/shell/CreateJobFunction.java > :100644 100644 54d8e9a... 51e189f... M > shell/src/main/java/org/apache/sqoop/shell/DeleteConnectionFunction.java > :100644 100644 6f07c32... ee1c8c4... M > shell/src/main/java/org/apache/sqoop/shell/DisableConnectionFunction.java > :100644 100644 9256712... 71295fa... M > shell/src/main/java/org/apache/sqoop/shell/EnableConnectionFunction.java > :100644 100644 6e5c9b5... dfaa90e... M > shell/src/main/java/org/apache/sqoop/shell/ShowConnectionFunction.java > :100644 100644 9a5386c... b69b726... M > shell/src/main/java/org/apache/sqoop/shell/ShowJobFunction.java > :100644 100644 8fa1bda... 791cc03... M > shell/src/main/java/org/apache/sqoop/shell/UpdateConnectionFunction.java > :100644 100644 b060bb4... 43baf28... M > shell/src/main/java/org/apache/sqoop/shell/UpdateJobFunction.java > :100644 100644 475f41c... 5765260... M > shell/src/main/java/org/apache/sqoop/shell/core/Constants.java > :100644 100644 56e0b4e... c2f9e22... M > shell/src/main/java/org/apache/sqoop/shell/utils/FormDisplayer.java > :100644 100644 c491ae5... e4b63ee... M > shell/src/main/java/org/apache/sqoop/shell/utils/FormFiller.java > :100644 100644 aa118e1... 7bc6154... M > shell/src/main/java/org/apache/sqoop/shell/utils/JobDynamicFormOptions.java > :100644 100644 2becc56... 7d92ef6... M > spi/src/main/java/org/apache/sqoop/connector/spi/SqoopConnector.java > :100644 000000 cdaa623... 0000000... D > spi/src/main/java/org/apache/sqoop/job/etl/Exporter.java > :000000 100644 0000000... 9b8d76f... A > spi/src/main/java/org/apache/sqoop/job/etl/From.java > :100644 000000 d4c9e70... 0000000... D > spi/src/main/java/org/apache/sqoop/job/etl/Importer.java > :000000 100644 0000000... a791945... A > spi/src/main/java/org/apache/sqoop/job/etl/To.java > :100644 100644 cf0b4aa... 9b791f8... M > spi/src/main/java/org/apache/sqoop/validation/Validator.java > :100644 100644 6fc485b... 54071f2... M > submission/mapreduce/src/main/java/org/apache/sqoop/submission/mapreduce/MapreduceSubmissionEngine.java > > > Diffs > ----- > > client/src/main/java/org/apache/sqoop/client/SqoopClient.java 05ea6d6 > common/src/main/java/org/apache/sqoop/common/ConnectorType.java > PRE-CREATION > common/src/main/java/org/apache/sqoop/json/ConnectorBean.java cbe049a > common/src/main/java/org/apache/sqoop/json/FrameworkBean.java eb79f98 > common/src/main/java/org/apache/sqoop/json/JobBean.java 1555bd5 > common/src/main/java/org/apache/sqoop/json/JobValidationBean.java > PRE-CREATION > common/src/main/java/org/apache/sqoop/model/MConnector.java 43fad27 > common/src/main/java/org/apache/sqoop/model/MFramework.java c742459 > common/src/main/java/org/apache/sqoop/model/MJob.java 849168d > common/src/main/java/org/apache/sqoop/model/MJobForms.java f697023 > > connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcConnector.java > e0da80f > > connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcValidator.java > 0c5f6e1 > > connector/connector-mysql-jdbc/src/main/java/org/apache/sqoop/connector/mysqljdbc/MySqlJdbcConnector.java > 17215f0 > core/src/main/java/org/apache/sqoop/connector/ConnectorHandler.java b80de7f > core/src/main/java/org/apache/sqoop/framework/ExecutionEngine.java f43942d > core/src/main/java/org/apache/sqoop/framework/FrameworkManager.java 505121c > core/src/main/java/org/apache/sqoop/framework/FrameworkValidator.java > f5f6a36 > core/src/main/java/org/apache/sqoop/framework/JobManager.java e052584 > core/src/main/java/org/apache/sqoop/framework/SubmissionRequest.java > 53d0039 > > core/src/main/java/org/apache/sqoop/framework/configuration/JobConfiguration.java > PRE-CREATION > core/src/main/java/org/apache/sqoop/repository/Repository.java ecf5004 > > execution/mapreduce/src/main/java/org/apache/sqoop/execution/mapreduce/MapreduceExecutionEngine.java > 5c0a027 > execution/mapreduce/src/main/java/org/apache/sqoop/job/JobConstants.java > 7fd9a01 > > execution/mapreduce/src/main/java/org/apache/sqoop/job/mr/ConfigurationUtils.java > bd11323 > > execution/mapreduce/src/main/java/org/apache/sqoop/job/mr/SqoopDestroyerExecutor.java > e1a95a7 > > execution/mapreduce/src/main/java/org/apache/sqoop/job/mr/SqoopInputFormat.java > 6891258 > execution/mapreduce/src/main/java/org/apache/sqoop/job/mr/SqoopMapper.java > 92de37e > > execution/mapreduce/src/main/java/org/apache/sqoop/job/mr/SqoopOutputFormatLoadExecutor.java > 7dedee9 > > repository/repository-derby/src/main/java/org/apache/sqoop/repository/derby/DerbyRepositoryHandler.java > 5bce3a9 > > repository/repository-derby/src/main/java/org/apache/sqoop/repository/derby/DerbySchemaConstants.java > fcbb475 > > repository/repository-derby/src/main/java/org/apache/sqoop/repository/derby/DerbySchemaQuery.java > 7042a53 > server/src/main/java/org/apache/sqoop/handler/JobRequestHandler.java > 362ba79 > shell/src/main/java/org/apache/sqoop/shell/CloneConnectionFunction.java > 9cce17b > shell/src/main/java/org/apache/sqoop/shell/CloneJobFunction.java f80552c > shell/src/main/java/org/apache/sqoop/shell/CreateJobFunction.java 598adbc > shell/src/main/java/org/apache/sqoop/shell/DeleteConnectionFunction.java > 54d8e9a > shell/src/main/java/org/apache/sqoop/shell/DisableConnectionFunction.java > 6f07c32 > shell/src/main/java/org/apache/sqoop/shell/EnableConnectionFunction.java > 9256712 > shell/src/main/java/org/apache/sqoop/shell/ShowConnectionFunction.java > 6e5c9b5 > shell/src/main/java/org/apache/sqoop/shell/ShowJobFunction.java 9a5386c > shell/src/main/java/org/apache/sqoop/shell/UpdateConnectionFunction.java > 8fa1bda > shell/src/main/java/org/apache/sqoop/shell/UpdateJobFunction.java b060bb4 > shell/src/main/java/org/apache/sqoop/shell/core/Constants.java 475f41c > shell/src/main/java/org/apache/sqoop/shell/utils/FormDisplayer.java 56e0b4e > shell/src/main/java/org/apache/sqoop/shell/utils/FormFiller.java c491ae5 > shell/src/main/java/org/apache/sqoop/shell/utils/JobDynamicFormOptions.java > aa118e1 > spi/src/main/java/org/apache/sqoop/connector/spi/SqoopConnector.java > 2becc56 > spi/src/main/java/org/apache/sqoop/job/etl/Exporter.java cdaa623 > spi/src/main/java/org/apache/sqoop/job/etl/From.java PRE-CREATION > spi/src/main/java/org/apache/sqoop/job/etl/Importer.java d4c9e70 > spi/src/main/java/org/apache/sqoop/job/etl/To.java PRE-CREATION > spi/src/main/java/org/apache/sqoop/validation/Validator.java cf0b4aa > > submission/mapreduce/src/main/java/org/apache/sqoop/submission/mapreduce/MapreduceSubmissionEngine.java > 6fc485b > > Diff: https://reviews.apache.org/r/23531/diff/ > > > Testing > ------- > > N/A > > This patch should break just about every thing. What needs to be verified is > that the interface is good and the models are fine. > > > Thanks, > > Abraham Elmahrek > >
