[
https://issues.apache.org/jira/browse/SQOOP-1666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196640#comment-14196640
]
Jarek Jarcec Cecho commented on SQOOP-1666:
-------------------------------------------
Comparing
{{[SqoopDestroyerExecutor|https://github.com/apache/sqoop/blob/sqoop2/execution/mapreduce/src/main/java/org/apache/sqoop/job/mr/SqoopDestroyerExecutor.java]}}
and
{{[JobManager|https://github.com/apache/sqoop/blob/sqoop2/core/src/main/java/org/apache/sqoop/driver/JobManager.java#L505]}}
code that executes the destroyer, it seems to me that the main difference is
in the way we get all the parameters. The Executor runs inside mapreduce job so
it needs to load all of them from jobconf whereas the version in {{JobManager}}
runs directly in server and hence loading all parameters from jobconf is not
necessary. One thing that stands out is that the Executor performs schema
matching that is not done in {{JobManager}} that we should probably fix (even
though it's probably not super critical).
> Sqoop2: JobManager code on failure should call SqoopDestroyerExecutor
> ----------------------------------------------------------------------
>
> Key: SQOOP-1666
> URL: https://issues.apache.org/jira/browse/SQOOP-1666
> Project: Sqoop
> Issue Type: Bug
> Reporter: Veena Basavaraj
> Assignee: Veena Basavaraj
> Fix For: 1.99.5
>
>
> There is more to just calling the destroyer.destroy method in the
> SqoopDestroyerExecutor
> Should not the code in JobManger call this utility class?
> The matching code is missing in the JobManager
> {code}
> public static void executeDestroyer(boolean success, Configuration
> configuration, Direction direction) {
> String destroyerPropertyName, prefixPropertyName;
> switch (direction) {
> default:
> case FROM:
> destroyerPropertyName = MRJobConstants.JOB_ETL_FROM_DESTROYER;
> prefixPropertyName = MRJobConstants.PREFIX_CONNECTOR_FROM_CONTEXT;
> break;
> case TO:
> destroyerPropertyName = MRJobConstants.JOB_ETL_TO_DESTROYER;
> prefixPropertyName = MRJobConstants.PREFIX_CONNECTOR_TO_CONTEXT;
> break;
> }
> Destroyer destroyer = (Destroyer)
> ClassUtils.instantiate(configuration.get(destroyerPropertyName));
> if(destroyer == null) {
> LOG.info("Skipping running destroyer as non was defined.");
> return;
> }
> // Objects that should be pass to the Destroyer execution
> PrefixContext subContext = new PrefixContext(configuration,
> prefixPropertyName);
> Object configConnection =
> MRConfigurationUtils.getConnectorConnectionConfig(direction, configuration);
> Object configJob = MRConfigurationUtils.getConnectorJobConfig(direction,
> configuration);
> // Propagate connector schema in every case for now
> Matcher matcher = MatcherFactory.getMatcher(
> MRConfigurationUtils.getConnectorSchema(Direction.FROM,
> configuration),
> MRConfigurationUtils.getConnectorSchema(Direction.TO, configuration));
> Schema schema = direction == Direction.FROM ?
> matcher.getFromSchema() : matcher.getToSchema();
> DestroyerContext destroyerContext = new DestroyerContext(subContext,
> success, schema);
> LOG.info("Executing destroyer class " + destroyer.getClass());
> destroyer.destroy(destroyerContext, configConnection, configJob);
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
