[
https://issues.apache.org/jira/browse/SQOOP-2496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alvin Ge updated SQOOP-2496:
----------------------------
Attachment: MapreduceSubmissionEngine.java
this...
> Sqoop2: Provide a way to inject external connectors
> ---------------------------------------------------
>
> Key: SQOOP-2496
> URL: https://issues.apache.org/jira/browse/SQOOP-2496
> Project: Sqoop
> Issue Type: Bug
> Affects Versions: 1.99.6
> Reporter: Jarek Jarcec Cecho
> Fix For: 1.99.7
>
> Attachments: MapreduceSubmissionEngine.java
>
>
> On internal hackathon we we're hacking Sqoop 2 connector with [~singhashish]
> and we went through few troubles that we should address.
> We have a a [configuration
> property|https://github.com/apache/sqoop/blob/sqoop2/dist/src/main/server/conf/sqoop.properties#L173]
> for extra directory from which we will load jar files. We were able to use
> this configuration property to load our hacked connector to Server, but we
> were not able to get it working through job submission. Here is the exception
> that we hit:
> {code}
> java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative
> path in absolute URI:
> jar:file://var/lib/sqoop/connectors/Connector-1.0-SNAPSHOT.jar!
> at org.apache.hadoop.fs.Path.initialize(Path.java:206)
> at org.apache.hadoop.fs.Path.<init>(Path.java:172)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.copyAndConfigureFiles(JobSubmitter.java:215)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.copyAndConfigureFiles(JobSubmitter.java:390)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:483)
> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1306)
> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1303)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
> at org.apache.hadoop.mapreduce.Job.submit(Job.java:1303)
> at
> org.apache.sqoop.submission.mapreduce.MapreduceSubmissionEngine.submitToCluster(MapreduceSubmissionEngine.java:274)
> at
> org.apache.sqoop.submission.mapreduce.MapreduceSubmissionEngine.submit(MapreduceSubmissionEngine.java:255)
> at org.apache.sqoop.driver.JobManager.start(JobManager.java:288)
> at
> org.apache.sqoop.handler.JobRequestHandler.startJob(JobRequestHandler.java:380)
> at
> org.apache.sqoop.handler.JobRequestHandler.handleEvent(JobRequestHandler.java:116)
> at
> org.apache.sqoop.server.v1.JobServlet.handlePutRequest(JobServlet.java:96)
> at
> org.apache.sqoop.server.SqoopProtocolServlet.doPut(SqoopProtocolServlet.java:79)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:646)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:723)
> {code}
> To put into nutshell,
> [ClassUtils.jarForClass|https://github.com/apache/sqoop/blob/sqoop2/common/src/main/java/org/apache/sqoop/utils/ClassUtils.java#L136]
> returns for external connectors path starting with prefix {{jar:file}} and
> suffix {{!}} that breaks mapreduce code.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
