Jarek Jarcec Cecho created SQOOP-2494:
-----------------------------------------
Summary: Sqoop2: Improve resiliency for malformed connectors
Key: SQOOP-2494
URL: https://issues.apache.org/jira/browse/SQOOP-2494
Project: Sqoop
Issue Type: Bug
Affects Versions: 1.99.6
Reporter: Jarek Jarcec Cecho
Fix For: 1.99.7
On internal hackathon we we're hacking Sqoop 2 connector with [~singhashish]
and we went through few troubles that we should address.
For example any exception that is generated during connector initialization
will cause Sqoop 2 to fail start. I think that being resilient and simply
ignore malformed connectors is much better approach.
Here is exception that put down Sqoop 2 server if we incorrectly marked
configuration objects:
{code}
2015-08-14 11:34:59,960 ERROR org.apache.sqoop.core.SqoopServer: Server startup
failure
org.apache.sqoop.common.SqoopException: MODEL_003:Object is not valid
configuration object - Missing annotation ConfigurationClass on class
com.cloudera.sqoop.Internal.conf.FromJobConfiguration
at org.apache.sqoop.model.ConfigUtils.toConfigs(ConfigUtils.java:78)
at org.apache.sqoop.model.ConfigUtils.toConfigs(ConfigUtils.java:67)
at
org.apache.sqoop.connector.ConnectorHandler.<init>(ConnectorHandler.java:95)
at
org.apache.sqoop.connector.ConnectorManager.initialize(ConnectorManager.java:177)
at
org.apache.sqoop.connector.ConnectorManager.initialize(ConnectorManager.java:155)
at org.apache.sqoop.core.SqoopServer.initialize(SqoopServer.java:57)
at
org.apache.sqoop.server.ServerInitializer.contextInitialized(ServerInitializer.java:41)
at
org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4210)
at
org.apache.catalina.core.StandardContext.start(StandardContext.java:4709)
at
org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:802)
at
org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:779)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:583)
at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:944)
at
org.apache.catalina.startup.HostConfig.deployWARs(HostConfig.java:779)
at
org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:505)
at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1322)
at
org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:325)
at
org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:142)
at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1068)
at org.apache.catalina.core.StandardHost.start(StandardHost.java:822)
at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1060)
at
org.apache.catalina.core.StandardEngine.start(StandardEngine.java:463)
at
org.apache.catalina.core.StandardService.start(StandardService.java:525)
at
org.apache.catalina.core.StandardServer.start(StandardServer.java:759)
at org.apache.catalina.startup.Catalina.start(Catalina.java:595)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289)
at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414)
2015-08-14 11:34:59,963 ERROR org.apache.sqoop.server.ServerInitializer: Sqoop
server failed to start
{code}
Or random exception when getting configuration classes:
{code}
2015-08-14 14:39:09,488 ERROR org.apache.sqoop.core.SqoopServer: Server startup
failure
org.apache.sqoop.common.SqoopException: EXCEPTION_001:Unknown job type - TO
at
com.cloudera.sqoop.Internal.Connector.getJobConfigurationClass(Connector.java:53)
at
org.apache.sqoop.connector.ConnectorHandler.<init>(ConnectorHandler.java:100)
at
org.apache.sqoop.connector.ConnectorManager.initialize(ConnectorManager.java:177)
at
org.apache.sqoop.connector.ConnectorManager.initialize(ConnectorManager.java:155)
at org.apache.sqoop.core.SqoopServer.initialize(SqoopServer.java:57)
at
org.apache.sqoop.server.ServerInitializer.contextInitialized(ServerInitializer.java:41)
at
org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4210)
at
org.apache.catalina.core.StandardContext.start(StandardContext.java:4709)
at
org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:802)
at
org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:779)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:583)
at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:944)
at
org.apache.catalina.startup.HostConfig.deployWARs(HostConfig.java:779)
at
org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:505)
at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1322)
at
org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:325)
at
org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:142)
at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1068)
at org.apache.catalina.core.StandardHost.start(StandardHost.java:822)
at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1060)
at
org.apache.catalina.core.StandardEngine.start(StandardEngine.java:463)
at
org.apache.catalina.core.StandardService.start(StandardService.java:525)
at
org.apache.catalina.core.StandardServer.start(StandardServer.java:759)
at org.apache.catalina.startup.Catalina.start(Catalina.java:595)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289)
at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414)
2015-08-14 14:39:09,492 ERROR org.apache.sqoop.server.ServerInitializer: Sqoop
server failed to start
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
