[
https://issues.apache.org/jira/browse/SQOOP-3087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15787890#comment-15787890
]
Steve Senior commented on SQOOP-3087:
-------------------------------------
I discovered that there is a workaround to this and that is to use the
{{--class-name}} Sqoop parameter. I'm not sure that this is the intended use
case for that parameter, but it produces the desired result.
If I set for e.g. {{--class-name=SH.TEST}} then the Avro schema fields are set
to {{name=TEST}} and {{namespace=SH}} which means we can overcome the exception.
> Dollar ($) in Oracle schema name causes sqoop to avro to throw exception
> ------------------------------------------------------------------------
>
> Key: SQOOP-3087
> URL: https://issues.apache.org/jira/browse/SQOOP-3087
> Project: Sqoop
> Issue Type: Bug
> Affects Versions: 1.4.6
> Reporter: Steve Senior
>
> Created an Oracle table called {{SH_$.TEST}} and attempt to Sqoop import as
> Avro ({{--as-avrodatafile}}) throws:
> {code}
> 16/12/21 16:54:58 ERROR sqoop.Sqoop: Got exception running Sqoop:
> org.apache.avro.SchemaParseException: Illegal character in: SH_$_TEST
> org.apache.avro.SchemaParseException: Illegal character in: SH_$_TEST
> at org.apache.avro.Schema.validateName(Schema.java:1142)
> at org.apache.avro.Schema.access$200(Schema.java:80)
> at org.apache.avro.Schema$Name.<init>(Schema.java:483)
> at org.apache.avro.Schema.createRecord(Schema.java:160)
> at
> org.apache.sqoop.orm.AvroSchemaGenerator.generate(AvroSchemaGenerator.java:97)
> at
> org.apache.sqoop.mapreduce.DataDrivenImportJob.generateAvroSchema(DataDrivenImportJob.java:154)
> at
> org.apache.sqoop.mapreduce.DataDrivenImportJob.configureMapper(DataDrivenImportJob.java:92)
> at
> org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:260)
> at org.apache.sqoop.manager.SqlManager.importTable(SqlManager.java:673)
> at
> org.apache.sqoop.manager.oracle.OraOopConnManager.importTable(OraOopConnManager.java:284)
> at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:507)
> at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:615)
> at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
> at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
> {code}
> I believe this is because it is trying to use the value {{SH_$_TEST}} for the
> {{"name"}} field in the Avro schema definition in the Avro data files. It
> seems that this is not allowed as per the Avro specification on names:
> https://avro.apache.org/docs/1.7.7/spec.html#Names
> Command line is:
> {code}
> sqoop import --connect jdbc:oracle:thin:@localhost:1521/ORA11204 --username
> xxx --password xxx --table SH_$.TEST --target-dir=/user/oracle/sh.db/test
> --delete-target-dir -m2 --direct --fetch-size=5000 --as-avrodatafile
> --outdir=/tmp
> {code}
> We know we can work around this by using the {{--query}} option for Sqoop,
> but this means we are not able to benefit from the OraOop optimisations with
> direct mode.
> Can functionality (such as a schema conversion parameter) be added to Sqoop
> to remove the {{$}} (and any other unsupported characters) from the {{name}}
> field stored in the Avro schema in the Avro data file?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
