[
https://issues.apache.org/jira/browse/SQOOP-1799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Veena Basavaraj updated SQOOP-1799:
-----------------------------------
Summary: Connector API : Ability for connector to indicate if its FROM and
TO support incremental reading/ writing (was: Ability for connector to
indicate if its FROM and TO support incremental reading/ writing)
> Connector API : Ability for connector to indicate if its FROM and TO support
> incremental reading/ writing
> ---------------------------------------------------------------------------------------------------------
>
> Key: SQOOP-1799
> URL: https://issues.apache.org/jira/browse/SQOOP-1799
> Project: Sqoop
> Issue Type: Sub-task
> Reporter: Veena Basavaraj
> Assignee: Veena Basavaraj
> Fix For: 1.99.5
>
>
> One suggestion would be have a connector's FROM/ TO initializer to expose if
> it even supports incremental. So this can be used to immediately validate the
> job creation.
> {code}
> sqoop > create incremental-job -f 1 -t 2
> {code}
> HDFS FROM supporting incrementation read ? Does this even apply. But surely
> the TO side should support the delta/ incremental write.
> Both the from connector and to connector has to support this feature before
> we proceed. The default will be false. The Initializer API will be updated to
> support this.
> {code}
> import java.util.LinkedList;
> import java.util.List;
> import org.apache.sqoop.schema.NullSchema;
> import org.apache.sqoop.schema.Schema;
> /**
> * This allows connector to define initialization work for execution,
> * for example, context configuration.
> */
> public abstract class Initializer<LinkConfiguration, JobConfiguration> {
> /**
> * Initialize new submission based on given configuration properties. Any
> * needed temporary values might be saved to context object and they will be
> * promoted to all other part of the workflow automatically.
> *
> * @param context Initializer context object
> * @param linkConfiguration link configuration object
> * @param jobConfiguration job configuration object for the FROM and TO
> * In case of the FROM initializer this will represent the FROM job
> configuration
> * In case of the TO initializer this will represent the TO job
> configuration
> */
> public abstract void initialize(InitializerContext context,
> LinkConfiguration linkConfiguration,
> JobConfiguration jobConfiguration);
> /**
> * Return list of all jars that this particular connector needs to operate
> on
> * following job. This method will be called after running initialize
> method.
> * @param context Initializer context object
> * @param linkConfiguration link configuration object
> * @param jobConfiguration job configuration object for the FROM and TO
> * In case of the FROM initializer this will represent the FROM job
> configuration
> * In case of the TO initializer this will represent the TO job
> configuration
> * @return
> */
> public List<String> getJars(InitializerContext context, LinkConfiguration
> linkConfiguration,
> JobConfiguration jobConfiguration) {
> return new LinkedList<String>();
> }
> /**
> * Return schema associated with the connector for FROM and TO
> * By default we assume a null schema. Override the method if there a
> custom schema to provide either for FROM or TO
> * @param context Initializer context object
> * @param linkConfiguration link configuration object
> * @param jobConfiguration job configuration object for the FROM and TO
> * In case of the FROM initializer this will represent the FROM job
> configuration
> * In case of the TO initializer this will represent the TO job
> configuration
> * @return
> */
> public Schema getSchema(InitializerContext context, LinkConfiguration
> linkConfiguration,
> JobConfiguration jobConfiguration) {
> return NullSchema.getInstance();
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
